TOBIAS HINZ

      Avatar

      Tech Lead

      Foundational GenAI Models

      Adobe Firefly

    I am the tech lead for core foundational models at Adobe Firefly, responsible for developing the next generation of Generative AI models. Previously, I was a Research Engineer at Adobe Research where I started the engineering efforts of Adobe Firefly and trained the first model that was released. Since then, I have led the development and training of the latest foundational imaging and video models powering Adobe Firefly's applications across various products and am working with teams across Adobe on improving these models to enable great user experiences and ship innovative applications.

    I obtained my PhD at the Knowledge Technology group at the University of Hamburg (Germany). Before that, I completed a research oriented Master's degree in Intelligent Adaptive Systems at the University of Hamburg. I received a Bachelor's degree in Business Informatics at the University of Mannheim, during which I also studied at the National University of Singapore for one semester.

News

  • 10/2024: I attended Adobe MAX where Adobe announced the public beta of our text-to-video model as well as GenExtend (powered by our video model) integrated in Premiere Pro.
  • 09/2024: A first sneak peek of our foundational text-to-video model at Adobe Firefly that we've been working on for the past year.
  • 06/2024: I attended CVPR 2024 in Seattle. Lots of great papers and conversations.
  • 03/2024: Our papers on Personalized T2I and Efficient Video Diffusion got accepted to CVPR 2024.
  • 11/2023: I have started leading the efforts on developing foundational text-to-video models for Adobe Firefly.
  • 10/2023: Our second generation of Firefly models was announced today at the Adobe MAX conference.
  • 08/2023: Our paper on diffusion model control was presented at SIGGRAPH.
  • 07/2023: I received the Adobe Tech Excellence Award, an annual recognition of outstanding technical talent for starting the engineering efforts of Adobe's Firefly models, writing more than 80% of the initial code base and serving as tech lead for foundational models.
  • 06/2023: Our paper on inpainting was presented as a highlight at CVPR.
  • 03/2023: Our Project Firefly was announced today.

    • Features I Worked On

    Image 2Adobe Firefly Video Model
    October 2024
    Oversaw technical development of the video model including training of the models and data preparation.    		Image 2Adobe Firefly Video GenExtend
    October 2024
    Consulted on training and development approaches.
    Image 2Adobe Firefly Image Model v2
    October 2023
    Developed and trained parts of the model; oversaw overall technical development of the core diffusion model.    		Image 1Adobe Firefly Image Model
    April 2023
    Started the effort that would ultimately result in the foundation of Adobe Firefly; developed and trained the core diffusion model.

    Selected Publications

    For a full list, have a look at my Google Scholar page.

        Personalized Residuals for Concept-Driven Text-to-Image Generation

        A novel and efficient approach for enabling personalized image generation with diffusion models.

        C. Ham, M. Fisher, J. Hays, N. Kolkin, Y. Liu, R. Zhang, T. Hinz, Conference on Computer Vision and Pattern Recognition 2024.

        Modulating Pretrained Diffusion Models for Multimodal Image Synthesis

        A multimodal conditioning module (MCM) for enabling conditional image synthesis using pretrained diffusion models.

        C. Ham, J. Hays, J. Lu, K. Singh, Z. Zhang, T. Hinz, SIGGRAPH Conference Proceedings 2023.

        SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model

        A diffusion model for shape-guided inpainting with better shape control and background preservation within the inpainted region.

        S. Xie, Z. Zhang, Z. Lin, T. Hinz, K. Zhang, Conference on Computer Vision and Pattern Recognition 2023.

        ASSET: Autoregressive Semantic Scene Editing with Transformers at High Resolutions

        A neural architecture for automatically modifying an input high-resolution image according to a user's edits on its semantic segmentation map.

        D. Liu, S. Shetty, T. Hinz, M. Fisher, R. Zhang, T. Park, E. Kalogerakis, ACM Transactions on Graphics (SIGGRAPH 2022) 2022.

        Improved Techniques for Training Single-Image GANs

        Improving the results and training speed of single-image GANs.

        T. Hinz, M. Fisher, O. Wang, S. Wermter, IEEE Winter Conference on Applications of Computer Vision 2021.

        Semantic Object Accuracy for Generative Text-to-Image Synthesis

        A novel GAN architecture and an improved metric to evaluate generative text-to-image synthesis models.

        T. Hinz, S. Heinrich, S. Wermter, IEEE Transactions on Pattern Analysis and Machine Intelligence 2020.

        Generating Multiple Objects at Spatially Distinct Locations

        Fine-grained control over the placement and identity of objects in images generated with a Generative Adversarial Network.

        T. Hinz, S. Heinrich, S. Wermter, International Conference on Learning Representations 2019.

        Speeding Up the Hyperparameter Optimization Of Deep Convolutional Neural Networks

        How to use lower dimensional data representations to speed up the hyperparameter optimization for CNNs processing images..

        T. Hinz, N. Navarro-Guerrero, S. Magg, S. Wermter, International Journal of Computational Intelligence and Applications 2018.


    See All
    See All

    Curriculum Vitae