Can We Still Beat a Robot in Accuracy and Reproducibility?

Man playing chess with robot
Photo: Pavel Danilyuk, Pexels.com

By: Juuso Vuorinen

Suomennos: Voiko ihminen vielä voittaa robotin tarkkuudessa ja toistettavuudessa?

As the academic year of 2023 reached its midpoint, many, including myself, were eagerly awaiting Christmas and the much-needed break from routines, not to mention all the delightful treats associated with the season. However, in the Computational Biology Group, we had one final mission to accomplish before tuning in to the season’s soothing choir chants and the melodic voice of Mariah Carey. We set out to explore whether humans, with or without prior experience, could still outperform laboratory automation robotics in terms of accuracy and reproducibility.

Juuso Vuorinen demonstrating the functions of the OT-2 liquid handler robot for the Computational Biology Group. Photo: Janette Hinkka

After experimenting with laboratory automation for some time, it became evident that, with a touch of human assistance, automation could effortlessly handle most of our routine tasks with remarkable accuracy and extreme reproducibility. Therefore, before it was too late, we were eager to determine whether an average bioinformatician could still outshine our OT-2 liquid transfer robot, or if the game was lost for us once and for all. The task was simple: could we pipette more accurately and reproducibility than our OT-2 robot?

One morning in December, we gathered our finest bioinformaticians, including our PI Matti Nykter, around a laboratory bench, with the noble aim of equipping every participant with the necessary knowledge and skills for the upcoming competition (Participants: Kaisa Aho, Matti Nykter, Lauri Ryyppö, Antti Kiviaho, Anssi Nurminen, Tuomo Virtanen, Sini Thusberg, Serafiina Jaatinen and Reetta Nätkin).

Ready for a challenge, we knew that while some of us had experience handling a pipette, not everyone had encountered such a device before. So, with the assistance of a custom designed training routine and our eye-of-the-tiger determination, we first honed our accuracy to its peak. The exactness and optimal training results we achieved owe much to the priceless efforts of our eagle-eyed, highly experienced employees—Janette Hinkka, Hanna Selin, and Päivi Martikainen—whom we wish to express our heartfelt gratitude to.

Lab personnel Hanna Selin and Päivi Martikainen instructing group members on correct pipetting technique. Participants training for the challenge and conducting the measurement manually, with their maximum precision. Photo: Janette Hinkka

After our brief training exercise, we were armed and ready for the challenge, proceeding to conduct the measurement with the Quant-iT dsDNA BR assay. Focusing on a level that seemed out of this world, participants gave their best to meet the challenge. The actual measurement workflow involved combining the Quant-iT reagent with a measurable analyte — in our case, a DNA sample — and then reading out the fluorescent signal from the dye within the reagent using a Tecan Spark plate reader. Following the measurement protocol carefully, we all clearly excelled at the task. Subsequently, it was merely a test of our newly programmed liquid handler’s functionality to repeat the process and attempt to outperform us. After analysis and plotting of the fluorescent signal, the results clearly illustrate how our accuracy compares to that of the liquid handler OT-2.

Results from the competition between bioinformaticians, experienced lab personnel, and the OT-2 liquid handling robot are as follows: Experienced laboratory personnel demonstrated the highest accuracy, with a mean of 102 ng/µl compared to the desired known concentration of 100 ng/μl (indicated by the dashed line). Their results also exhibited low standard deviation (Std Dev) at 4.16% and coefficient of variation (CV) at 4.51%. In contrast, the OT-2 robot secured victory in the competition for reproducibility with the lowest Std Dev of 3.94 and CV of 3.49%. Remarkably, the bioinformaticians showcased admirable resilience, achieving a nearly accurate mean concentration of 94.7 ng/µl. Furthermore, their exceptional performance extended to precision, reflected in a low CV of 8.71% and a Std Dev of 8.25. Image: Juuso Vuorinen

Lastly, for fun, I asked our friendly large language model OpenAI’s GPT-3.5 to draft a summary of the results of our endeavor. Here’s what GPT-3.5 had to say (with only slight modifications):

“🧬✨ Pipetting Olympics Results! 🏆✨

After a thrilling competition in the Lab Olympics, our teams showcased their pipetting prowess. Let’s dive into the results!

🔍 Bioinformatician Novices – The Adventurers:

Mean Concentration: 94.47 ng/µl    🎉 Despite a bit of a concentration adventure, our brave novices held their own! Their pipetting journey included some scenic detours, resulting in a CV of 8.69%. A valiant effort with a touch of unpredictability!

🤖 OT-2 Automation – The Robot Champions:

Mean Concentration: 114.17 ng/µl  🚀 Our robotic overlords delivered precision with an iron grip! The OT-2 team’s concentration is a shining beacon of accuracy, boasting a rock-solid CV of 2.85%. Robo-high-fives all around for this stellar performance although being off by a bit!

🧪 Experienced Pipette Masters – The Artisans:

Mean Concentration: 101.53 ng/µl 🌟 With years of pipetting finesse, our seasoned experts sculpted concentrations with grace. A moderate CV of 5.52% reflects their artistry, blending accuracy and experience. Masterclass in manual pipetting!

Overall Lab Harmony: All teams hit the bullseye near the aimed 100 ng/µl concentration! 🎯 Each group brought its unique flair to the competition, creating a vibrant spectrum of results. Cheers to the diverse skills in our pipetting palette! In conclusion, whether you’re a lab adventurer, a robot whisperer, or a pipette artisan, our Lab Olympics proved that precision comes in many forms. Let the pipetting celebration continue!”

Juuso Vuorinen, Research Assistant from the Computational Biology Reserach Group. Photo: Computational Biology Group

Notably, an exceptional measurement record of 100.07 ng/µl prompted a colleague to joke, “who is this god among us mortals.” Suspiciously enough, another coworker pinpointed an outstanding record of 122.44 ng/µl, raising suspicions of a possible infiltrated robot 🤖 among us. However, they both soon realised that in this dystopian future scenario, this developed test protocol could indeed serve as a replacement for the Voight-Kampff test. Functionality of our protocol in this application, however, is yet to be proven.

Lastly, the OT-2’s results, that we all likely noted being off by a bit, were tracked to a small programming error in the code for the assay. My newly written code didn’t explicitly instruct the robot on how and from which height to pipette, which likely caused it to overestimate the concentration of the sample. Without specific instructions the robot pipetted the sample from the bottom of the well without prior pipette mixing of the sample, which is very likely to cause the overestimation of our known DNA concentration. Our experimentation nicely highlights the necessity of empirical testing in development and iteration of new processes and programs. Many thanks to all participating in this experiment, you all made the assay better than it was before.

Want to learn more about the Finnish Center of Excellence in Tumor Genetics?
Keep up to date with the latest tweets, videos and blog posts for a peek into the everyday
life in cancer research; subscribe to our blog and YouTube channel!
and don’t forget to follow us on Twitter/X (@CoEinTG)!

One thought on “Can We Still Beat a Robot in Accuracy and Reproducibility?

Comments are closed.