Robotic Painting: Iterative Portrait Painting Using a Collaborative Robot

Project Overview:

This project aims to develop a system allowing a collaborative robot to iteratively paint portraits, enhancing software and hardware integration for artistic robot applications. The objective is to create a software stack that generates basic portraits and the design of a robust hardware setup, including implementing robot paths for paint and brush changing stations. By combining these two aspects, the project aims to advance robotic systems' capabilities in the creative portrait painting domain, enabling iterative refinement of artistic output.

The project with involve a Franka Emika Panda, a collaborative Robotic arm as primary hardware.

Software Development:

The software component of the project involves creating a system that processes input images of portraits and translates them into brushstroke instructions for the robot. This work will require designing algorithms that simplify the image into paintable elements, including edge detection and iterative layer refinement. The software stack should be capable of guiding the robot through multiple stages of painting, from rough sketching to detailed completion, with each iteration adding complexity and depth to the portrait. Users should be capable of intervening in the painting process to add additional features with a seamless resuming of the robot operation afterward. Key considerations include stroke order, brush selection, and user interaction affordances to ensure a smooth painting process.

Hardware Setup and Robot Design:

The hardware aspect of the project focuses on configuring the robot for practical use in a painting environment. A required element is implementing robot paths for already existing but not yet functioning components like brush changers, color palettes, and brush cleaners. This station should ensure seamless interaction with the robot, minimizing downtime and ensuring precision in the transitions.

Iterative Painting Process:

One of the project's core goals is to enable iterative painting, where the robot progressively refines its work over multiple passes with potential additional user input or direct painting. In each iteration, the software will analyze the current state of the portrait, identify areas for refinement, and adjust the next set of brushstroke instructions accordingly. This iterative process will allow for increased accuracy and detail with each cycle, mimicking how human artists layer their work. By integrating feedback from the painting process, the robot will continuously improve its output, balancing between structured guidelines from the software and the inherent unpredictability of physical painting.

Integration of Software and Hardware:

The project will emphasize the seamless software and hardware integration to create a cohesive system. This project includes working with image abstraction and portrait generation and optimizing the coordination between image processing and real-time robot control. We pay special attention to real-world challenges such as robot imprecision, hardware constraints, challenges in practical portrait abstraction, and stroke extraction.

Expected Outcomes:

The project's outcome will be a fully functional robotic system capable of painting portraits iteratively, with a dynamic paint and brush changing station. The system will demonstrate the ability to autonomously generate rudimentary portraits, refine them over multiple passes, and achieve a high level of artistic expression. This project will contribute to robotic art, offering insights into how iterative processes and human-robot interaction can be leveraged in creative tasks. The software stack and hardware design could serve as a foundation for further advancements in robotic painting and collaborative artistic creation.

Requirements:

General interest in image processing, robotics, working with portraits

Basic knowledge of Python programming, image processing

Willingness to take an artistic perspective

Large interest in working with a hardware setup

If you are interested, please contact Michael Stroh.