Work Experience

Joined the Paris office of Instadeep as a Reinforcement Learning Engineer while the company was still a small startup with around 10 people in the office and 80 people split across the whole company. Now Instadeep is one of the most successful startups in Europe, in 2023 getting acquired by BioNTech. It has over 100 people in Paris and almost a headcount of 500 people in the whole company and I was part of the core team that delivered several results that led to the success of the company. Additionally I switched offices from the Paris to Berlin and helped grew the office alongside one colleague from the two of us to over 20 people right now.

The main project where I have achieved state of the art results over the past 5 years was in our collaboration with DeutscheBahn. Here, in a team of 16 people using Gitlab and following Agile, I helped build a solution to the NP-hard problem of train scheduling. The framework we use is that of Multi-Objective Multi-Agent Reinforcement Learning. By leveraging an advanced simulator of realistic train physics, we trained a multi-agent system using algorithms such as APEX, PPO, STMARL, GAT, and POPPY. More details about our approach can be found in the paper we published in the 27th International Conference on Intelligent Transportation Systems in 2024. The are two outstanding results worth mentioning. First, at the end of 2021 we build an RL Agent that, while being trained on building schedules for configurations of 10-40 trains using a curriculum, was able to generalize in building a schedule with over 300 trains in a map resembling the Koln-Koblenz region. This remarkable result led to the extension of our collaboration with DB to 7 more years. Second,  we have built the AI Agent that was integrated into an internal product that was able to adjust schedules by minimizing the adjustments and delay in real-time if a disruption (such as blocking a track) was introduced on the railway infrastructure of Magdeburg.  My main responsibility was to lead the delivery of the AI Agent microservice and integrating it via Azure and Kubernetes into an internal product for train scheduling. The product is able to schedule hundreds of trains on real German railway infrastructures modelling the regions of Stuttgart or Magdeburg adjust schedules by minimizing the delay in real-time if a disruption was introduced on the railway infrastructure, such as a track being blocked. We built the microservice in Python using a PyTorch model, Ray, and containerized it using Docker. We trained the model using a Sebulba arhitecture using RLLib, Azure, and Kubernetes.

In parallel with the DB project I was part of the pre-sales discussions, managed, and delivered successfully on the objectives in the following projects: 

• Internship on the Marine Transportation Optimization problem of Empty Container Repositioning (ECR). I proposed, supervised and contributed technically to the Internship. Here we published a Medium article and a paper in IEEE Transactions on Intelligent Transporation Systems 2023.

• Internship  on the Marine Transportation Optimization problem of Network Design. I proposed, supervised and contributed technically to the Internship. Here a published a Medium article.

• Client Proof of Concept Project on Minimizing the Soot Emission with an Industrial Leader in Automotive. I was part of the pre-sales discussions, proposed the technical approach, lead the project, interacted constantly with the client, and delivered to the client.

• Client Proof of Concept Project on Warehouse Optimization with the largest manufacturing company in Europe. I was part of the pre-sales discussions, proposed the technical approach in JAX, lead the project, interacted constantly with the client, and delivered to the client.

• Client Proof of Concept for evaluating football games and players with a big German Club. I was part of the pre-sales discussions, proposed the technical approach, lead the project, interacted constantly with the client, and delivered to the client.

• Client MVP for optimizing the dispatching of personnel required

I was also the Hiring Lead in Engineering for Paris and Berlin and helped hire tens of talented people at Instadeep and now I am directly supervising 5 of them.

Finished an internal Team Lead course to sharpen my leading abilities.

Done the Generative Adversarial Network Specialization on Coursera.

Built a library in Python that predicted gas occurrences in underground environments by formulating the problem as a classification problem with imbalanced data. Used several Synthetic Minority Oversampling Techniques (SMOTE) alongside a Neural Network in Tensorflow 2.0 obtaining a Precision of 0.93 and Recall of 0.98.

Delivered a Proof of Concept (PoC) for Density Estimation of material formation in underground environments. By using a Neural Network or a Gradient Boosted Trees (XGBoost) Model I outperformed significantly the existing Physics-based Model. For explainability I used Shapley Additive Explanation (SHAP) features.

During a 4-week training program in a team of 15 I learned the Agile frameworks of Scrum and KanBan. To apply our knowle

Refactored the data format of the internal Reinforcement Learning (RL) Python library improving the storage size by a factor of 100.

Extended the internal library’s implementation of the Proxy Policy Optimization (PPO) and Advantage Actor Critic algorithms to work with discrete action space.

Implemented Deep Deterministic Policy Gradient into the library and fine-tuned it in the OpenAI Gym and the internal autonomous driving environment.

Worked on the "Learning from Demonstrations in the Wild" paper which got accepted at the International Conference on Robotics and Automation 2019.

Built a material identifier rig with 2 hyperspectral cameras and a spectrometer.

Designed and implemented a Python library for material capture and spectral signature computation. Built a database with 10 materials and thousands of images.

Implemented several Computer Vision classical approaches and a Deep Learning approach using a CNN written in tensorflow for material classification. The DL approach outperformed the rest and gave a 99.8% accuracy.

Developed a ROS (Robot Operating System) network to capture GPS points.

Wrote a ROS Package in C++ for autonomous driving of a tractor on the map from the GPS, using A* to find the most efficient route between the GPS waypoints.

Added a remote Arduino connected joystick for fine-grained intervention in the motor controller of the tractor.