ABOUT

portrait I am an undergraduate studying Electrical and Computer Engineering at the University of Texas at Austin. I am currently working on situational awareness and IoRT in the Nuclear and Applied Robotics Group, advised by Dr. Mitch Pryor.

I am broadly interested in the use of machine learning and the Internet of Things to improve decisional autonomy and adaptability in robotics.

When I'm not working on robots or embedded devices, I like to collect and tinker with antique electronics. I'm currently working on converting an original 1994 AppleDesign Keyboard to a wireless keyboard.

When my eyes need a break from staring at my computer display, I like to sketch or read. Feel free to check out my work using the links in the bottom right and reach out if you're interested.

Robotics

I believe robotics has massive potential as a tool to aid humanity across a huge number of fields, especially as we adjust to life in a post-COVID world. However, many robotic systems are unable to operate in the complex, ever-changing world that humans are so used to. I am interested in how to best improve the usability of robotic systems by leveraging technologies such as AI and IoT. My current projects are InSitu, a situational awareness package aiming to generalize sensor overlays for video feeds, and ThirdEye, a package intended to bridge IoT endpoints to ROS using FreeRTOS and AWS.

IoT and Embedded Systems

Embedded systems is a close second to robotics on my list of interests. Specifically, I am interested in the developmental challenges of large-scale IoT deployments. I'm working on an open-source IoT endpoint for my senior design project, under the mentorship of Professor Mark McDermott. The system is based around the ESP32 SoC by Espressif and running FreeRTOS. Check back soon for updates!

Computing

Computing and its relatively short but storied history is another one of my key interests. Inspired (rather, infuriated) by the tools used in my introductory computing course, I developed a Visual Studio Code extension and an accompanying assembler for the LC3 assembly language, which you can find on my Github. More recently, I've been working on my own Linux customizations, including a shellscript-based status bar and window manager extensions.

PROJECTS

InSitu: Extensible Situational Awareness for Supervised Autonomy

[Video] ROS World 2021 Lightning Talk

Situational awareness, the perception and comprehension of environmental factors, has long played a key role in decision-making across a broad range of applications. Many of these coincide with key roles for robotics, such as health care, emergency response, and offshore oil and nuclear plant management. Robots often carry a large array of sensors comprised of cameras, LiDARs, force-torque sensors, and other domain-specific sensors to provide situational awareness. However, many current robot control interfaces do not effectively present this information to remote operators, posing significant challenges in teleoperation and supervised autonomy.

To address these challenges, our group has developed InSitu, a ROS visualization framework for situational awareness. At its core, InSitu organizes image streams into task views and provides a rich plugin interface to overlay relevant task information, such as data from onboard or environmental sensors. InSitu is highly configurable and includes a set of basic overlays to provide common SA functionality. Video:

InSitu was developed to support ongoing projects in the Nuclear and Applied Robotics Group, such as our collaboration with Woodside to develop robotic oil plant maintenance capabilities. The screenshot in the upper right showcases InSitu displaying a simple task view, overlaying IoT data related to the valve turn. More generic applications include pointcloud overlays for navigation or joint limit warnings and force-torque arrows for manipulation. Check out the project Github page for a quick-start guide and a list of available overlays.

InSitu started as a small support project but has quickly grown into something we would like to share with other groups. There are several planned features in the development pipeline, including:

improvements to the task view editing interface
quality-of-life changes for users hoping to develop their own overlays
more included overlays so new users can get started with InSitu without writing a single line of code

If you have any feature ideas or find any bugs, please create an issue on the project Github.

e-Ink Graphing Calculator

[Video] Fall 2020 Embedded Systems Competition Runner Up

The handheld calculator market was once saturated with competitors, much like the current smartphone market, but has since become dominated by just a few companies; Texas Instruments, Hewlett Packard, and Casio. So for our Embedded Systems Lab final project, my teammates and I decided to light a fire under their asses by building our own graphing calculator. We wrote a custom driver for a 4.2" high definition e-Ink display that absolutely dwarfed the competition, and created the Embedded Arithmetic Shell Interface (EASI) to compete with the big boys' software offerings. Check out our presentation below:

We've decided to continue this project! Check out the project page and let us know what you think!

AirTouch

Fall 2020 Ram's Horn Award Overall Winner

Digital Image Processing was a hidden gem in an otherwise unremarkable slugfest of a semester. I learned a lot about how AI is applied in computer vision and where it is most effective. This was a small project that I worked on with two friends for our semester project. We used a Intel Realsense D455 depth camera to create a virtual drawing tablet. Here's our presentation video:

My teammates shot down my idea of calling hand candidates "handidates," but I'd like to share it for anyone doing similar work.

PUBLICATION*

Adam Pettinger, Cassidy Elliot, Pete Fan, and Mitch Pryor (2020).“Reducing the Teleoperator’s Cognitive Burden for Complex Contact Tasks Using Affordance Primitives”. In: Proceedings of IROS 2020. Las Vegas, USA. PDF

* Currently working on the 's'