TOBIAS HINZ

      Avatar

      Machine Learning

      Research Engineer

      Adobe Research

    I am the tech lead for core foundational models at Adobe Firefly, responsible for developing the next generation of Generative AI models. Previously, I was at Adobe Research where I started the engineering efforts of Adobe Firefly and trained the first model that was released. Since then, I have led the development and training of the latest foundational imaging models powering Adobe Firefly's applications across various products and am actively working with teams across Adobe on improving these models to achieve the best user experience and ship innovative applications.

    I obtained my PhD at the Knowledge Technology group at the University of Hamburg (Germany). Before that, I completed a research oriented Master's degree in Intelligent Adaptive Systems at the University of Hamburg. I received a Bachelor's degree in Business Informatics at the University of Mannheim, during which I also studied at the National University of Singapore for one semester.

News

  • 10/2023: Our second generation of Firefly models was announced today at the Adobe MAX conference.
  • 08/2023: Our paper on diffusion model control was presented at SIGGRAPH.
  • 07/2023: I received the Adobe Tech Excellence Award, an annual recognition of outstanding technical talent for starting the engineering efforts of Adobe's Firefly models, writing more than 80% of the initial code base and serving as tech lead for foundational models.
  • 06/2023: Our paper on inpainting was presented as a highlight at CVPR.
  • 03/2023: Our Project Firefly was announced today.
  • 05/2022: Our paper on transformers for image editing has been accepted to SIGGRAPH 2022.
  • 05/2022: I have moved to San Jose (California) to join the Adobe Headquarters office.
  • 01/2022: Our paper won the Best Application Paper award at WACV 2022.
  • 09/2021: Our review paper on adversarial text-to-image synthesis has been published by the Neural Networks Journal.
  • 08/2021: Check out this concert which combines music with generated videos. Our ConSinGAN approach was used in the video for "Robbie's Hobbies.

Selected Publications

For a full list, have a look at my Google Scholar page.

      Modulating Pretrained Diffusion Models for Multimodal Image Synthesis

      A multimodal conditioning module (MCM) for enabling conditional image synthesis using pretrained diffusion models.

      C. Ham, J. Hays, J. Lu, K. Singh, Z. Zhang, T. Hinz, SIGGRAPH Conference Proceedings 2023.

      SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model

      A diffusion model for shape-guided inpainting with better shape control and background preservation within the inpainted region.

      S. Xie, Z. Zhang, Z. Lin, T. Hinz, K. Zhang, Conference on Computer Vision and Pattern Recognition 2023.

      ASSET: Autoregressive Semantic Scene Editing with Transformers at High Resolutions

      A neural architecture for automatically modifying an input high-resolution image according to a user's edits on its semantic segmentation map.

      D. Liu, S. Shetty, T. Hinz, M. Fisher, R. Zhang, T. Park, E. Kalogerakis, ACM Transactions on Graphics (SIGGRAPH 2022) 2022.

      CharacterGAN: Few-Shot Keypoint Character Animation and Reposing

      BEST PAPER AWARD - Train a GAN on as few as 10 images of a given character for animation and reposing.

      T. Hinz, M. Fisher, O. Wang, E. Shechtman, S. Wermter, IEEE Winter Conference on Applications of Computer Vision 2022.

      Improved Techniques for Training Single-Image GANs

      Improving the results and training speed of single-image GANs.

      T. Hinz, M. Fisher, O. Wang, S. Wermter, IEEE Winter Conference on Applications of Computer Vision 2021.

      Semantic Object Accuracy for Generative Text-to-Image Synthesis

      A novel GAN architecture and an improved metric to evaluate generative text-to-image synthesis models.

      T. Hinz, S. Heinrich, S. Wermter, IEEE Transactions on Pattern Analysis and Machine Intelligence 2020.

      Generating Multiple Objects at Spatially Distinct Locations

      Fine-grained control over the placement and identity of objects in images generated with a Generative Adversarial Network.

      T. Hinz, S. Heinrich, S. Wermter, International Conference on Learning Representations 2019.

      Speeding Up the Hyperparameter Optimization Of Deep Convolutional Neural Networks

      How to use lower dimensional data representations to speed up the hyperparameter optimization for CNNs processing images..

      T. Hinz, N. Navarro-Guerrero, S. Magg, S. Wermter, International Journal of Computational Intelligence and Applications 2018.


See All
See All

Curriculum Vitae