Michael Dorkenwald

Michael Dorkenwald

PhD Student

University of Amsterdam

About me

I am PhD student in the QUVA lab at the University of Amsterdam supervised by Yuki Asano and Cees Snoek. I am also part of the ELLIS PhD program in cooperation with Qualcomm. My research primarily revolves around video data, which provides a lens to the intricate dynamics and mechanisms that define our world. I am interested in self-supervised methods and vision-language models as these yield more generalizable visual representations.

Before, I received my master’s degree in physics from Heidelberg University during which I was part of the research group from Björn Ommer. There, I was working on understanding human and object dynamics within generative frameworks primarily for video synthesis. I had the opportunity for a research visit at Kosta Derpanis’s lab in Toronto. Furthermore, I completed an internship in the AWS Rekognition team where I worked on self-supervised video representation learning with Davide Modolo and Josephe Tighe.

News

Publications

PIN: Positional Insert Unlocks Object Localisation Abilities in VLMs
Accepted to CVPR 2024
We unlock the object localisation capabilities of caption-based VLMs with PIN, a trainable spatial prompt integrated into the frozen VLM, without requiring labeled detection data, demonstrating strong zero-shot localisation across various image domains.
Michael Dorkenwald, Nimrod Barazani, Cees Snoek, Yuki Asano
PDF Cite Project Page
PIN: Positional Insert Unlocks Object Localisation Abilities in VLMs
SCVRL: Shuffled Contrastive Video Representation Learning
CVPR 2022 I3D-IVU workshop
We propose SCVRL, a novel contrastive-based framework for self-supervised learning for videos. Differently from previous contrast learning based methods that mostly focus on learning visual semantics (e.g., CVRL), SCVRL is capable of learning both semantic and motion patterns.
Michael Dorkenwald, Fanyi Xiao, Biagio Brattoli, Joseph Tighe, Davide Modolo
PDF Cite Project Page
SCVRL: Shuffled Contrastive Video Representation Learning
iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesis
ICCV 2021
We present iPOKE, a model for locally controlled, stochastic video synthesis based on poking a single pixel in a static scene, that enables users to animate still images only with simple mouse drags.
Andreas Blattmann, Timo Milbich, Michael Dorkenwald, Bjoern Ommer
PDF Cite Code Project Page
iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesis
Stochastic Image-to-Video Synthesis using cINNs
CVPR 2021
We present a framework for both controlled and stochastic image-to-video synthesis. We bridge the gap between the image and video domain using conditional invertible neural networks and account for the inherent ambiguity with a dedicated, learned scene dynamics representation.
Michael Dorkenwald, Timo Milbich, Andreas Blattmann, Robin Rombach, Konstantinos G. Derpanis, Björn Ommer
PDF Cite Code Project Page
Stochastic Image-to-Video Synthesis using cINNs
Behavior-Driven Synthesis of Human Dynamics
CVPR 2021
We present a model for human motion synthesis which learns a dedicated representation of human dynamics independent of postures. Using this representation, we are able to change the behavior of a person depicted in an arbitrary posture or to even directly transfer behavior observed in a given video sequence.
Andreas Blattmann, Timo Milbich, Michael Dorkenwald, Bjoern Ommer
PDF Cite Code Project Page
Behavior-Driven Synthesis of Human Dynamics
Understanding Object Dynamics for Interactive Image-to-Video Synthesis
CVPR 2021
We propose an approach for interactive image-to-video synthesis that learns to understand the relations between the distinct body parts of articulated objects from unlabeled video data, thus enabling synthesis of videos showing natural object dynamics as responses to local interactions.
Andreas Blattmann, Timo Milbich, Michael Dorkenwald, Bjoern Ommer
PDF Cite Code Project Page
Understanding Object Dynamics for Interactive Image-to-Video Synthesis
Unsupervised behaviour analysis and magnification (uBAM) using deep learning
Nature Machine Intelligence
Unsupervised behaviour analysis and magnification (uBAM), an automatic deep learning algorithm for analysing behaviour by discovering and magnifying deviations.
Biagio Brattoli, Uta Buechler, Michael Dorkenwald, Philipp Reiser, Lineard Filli, Fritjof Helmchen, Anna-Sophia Wahl, Bjoern Ommer
PDF Cite Code Project Page
Unsupervised behaviour analysis and magnification (uBAM) using deep learning
Unsupervised Magnification of Posture Deviations across Subjects
CVPR 2020
An approach to unsupervised magnification of posture differences across individuals despite large deviations in appearance.
Michael Dorkenwald, Uta Buechler, Bjoern Ommer
PDF Cite Video Project Page
Unsupervised Magnification of Posture Deviations across Subjects