<h2>Description</h2>
<p>Between the August of 2022 and the March of 2024, I had the pleasure of working on the beta for the Xperiential Robotics Platform (XRP). The XRP Robot is a low-cost robots kit <em><a href="https://www.sparkfun.com/products/22230">($65 for educators, $115 for hobbyists)</a></em>
designed predominantly for classroom use. This robot is equipped with a Raspberry Pi Pico, and can be programmed in C/C++, <em><a href="https://micropython.org/">MicroPython</a></em>, and even <em><a href="https://docs.wpilib.org/en/stable/docs/xrp-robot/index.html">Java, using the WPILib integration</a></em>. I was the software lead for the MicroPython library, which is the main one used in classrooms.</p>
<h3>Methodology</h3>
<p>With the goal of making this library intuitive, the first thing we did is draft up the class structure. This was done informally and on paper, but above is a recreation of this as a UML diagram. We decided early that composition was preferable to inheritance, so an EncodedMotor contains both an Encoder and a Motor, instead of EncodedMotor being a type of Motor with bonus encoder functionality. From there, it was just a matter of replicating this thought process all the way down the line. This resulted in a Drivetrain class, which contains both the left and right drive motors, as well as the Inertial Measurement Unit (IMU). The advantage of a structure like this, especially early on when hardware components were not set in stone, was that we could swap out implementations for motors, encoders, and the IMU, all with minimal changes to the library.</p>
<p>Once we decided on the path forwards with object structure within XRPLib, and discussed the most intuitive behavior for each component, we began on implementation. From one of the Unified Robotics classes at WPI, I knew we could track encoder positions using an Interrupt Service Routine (ISR). MicroPython makes the implementation for this pretty easy, so I set that up and get a very basic driving program to run. It sucks. It turns out, a big drawback of a language like Python is that, as an interpreted language, code execution is not fast. As such, the multitude of interrupts resulting from normal driving speeds were eating up a large amount of time, making my robot stop later than it should have, and overall having a large amount of variance between what should have been evenly spaced prints.</p>
<p>It turns out, the Raspberry Pi Pico had our solution. The <em><a href="https://www.raspberrypi.com/documentation/microcontrollers/silicon.html#rp2040">rp2040 processor</a></em> on the Pico has assembly level state machines (PIOs) that can be used to process basic io operations. We could use this to track the two encoder pins and increment the encoder position accordingly, which reduces the execution time of this loop to effectively nothing, as the PIO was not directly connected to the CPU, and would run perfectly synchronously for up to four motors. Unfortunately, that is not the last we'd hear of interrupts causing performance issues.</p>
<p>I wrote a <em><a href="https://en.wikipedia.org/wiki/Proportional%E2%80%93integral%E2%80%93derivative_controller">Proportional-Integral-Derivative (PID) controller</a></em> for controlling the position and speed of the motors as they drive. I moved this controller into a new class (<code>pid.py</code>) to be used anywhere I wanted, as the backing math is not implementation-specific. However, to effectively control the speed of the motors, timer-based interrupts are a critical feature. This time, lukily, I could control the rate that they get called, so I set the rate to 50Hz, which was unobtrusive.</p>
<p>After finishing the drivetrain implementation, I turned to focus on sensors. The XRP Robot ships with 4 sensors: An Ultrasonic Distance Sensor, Two Reflectance Sensors (which allow the robot to follow a line), and an IMU (which contains an accelerometer and a gyroscope). I had worked with all of these sensors before, but only in C++, and through other libraries or codebases that already had the communication with these sensors written. Thankfully, I'm a quick learner (and had access to the C++ versions of the code for the ultrasonic and gyroscope).</p>
<p>The IMU required me to learn how to implement I2C communication in order to talk to it, and once I did I needed to implement functionality for all of the register read/writes necessary to configure it the way I wanted. Trying to package the library to keep the code readable and beginner friendly has been quite a challenge, but the solution we landed on (me, in conjunction with an engineer from Sparkfun) with was to split imu.py into two files, one for constants and register-bindings, and one for the actual IMU class.</p>
<h3>In Hindsight (Nov 2024 Retrospective)</h3>
<p>When I started this project, the goal was to make a library that worked pretty quickly, such that we could start using an Agile structure with good git practices. As such, we started making the Micropython library using MicroPython, instead of utilizing the underlying C code to obtain better execution times. This caused some issues later, particularly when it came to interrupts, where fast execution times are essential to not disrupting performance in other segments of the code. Unfortunately, by the time I stopped leading this part of the project, a C-based implementation of XRPLib still had not been started. I may, at some point, return to the project to push for this better implementation.</p>
<h3>Going Forward (Updates Feb 2025)</h3>
<p>I started working on XRPLib again in January 2025. The project was readying to move out of Beta, planned to begin production mid February. There was a new control board, powered by the new <em><a href="https://www.raspberrypi.com/documentation/microcontrollers/silicon.html#rp2350">rp2350 processor</a></em>, with new motor drivers and features not present on the Beta. There were issues I wanted to fix from the previous iteration of XRPLib, but more importantly was the new design challenge: How to add new features, while keeping all prior user code identical in functionality, while keeping one library that worked on both versions of the XRP Control Board.</p>


XRPLib - eXperiential Robotics Platform

<h3>Project Overview</h3>
<p>Roadside trash, a serious environmental and economic issue, is difficult, dangerous, and expensive to remove. The Terrawarden drone, a one-meter-wide quadrotor with a 3-DOF manipulator and compliant gripper, addresses the challenges associated with collecting trash on the inaccessible and uneven terrain of highway medians by autonomously detecting and retrieving trash using on-board perception.</p>
<h3>Final Results</h3>
<p>In the end, our drone was able to take off, identify a can, navigate to it and grasp it with the 3-DOF arm. However, we found there was a limiting balance for the weight of a can, as they would blow away if the drone was nearby if too light, but be unable to be lifted if they were too heavy. Due to time constraints, a solution for this was not developed, although a PhD student in our advisor's lab was given this drone to be the basis for a potential thesis paper following our work on the project.</p>
<p><strong>Metrics:</strong></p>
<ul>
<li>Autonomous flight and trash pickup​</li>
<li>Flight hold within 10 cm of setpoint​</li>
<li>Manipulator within 2 cm of setpoint​</li>
<li>Gripper holds 350 g payload​</li>
<li>Trash detection 83.2% accurate​ (number of cans detected / number of total predictions)
<ul>
<li>39.1% of real-world cans missed​</li>
</ul>
</li>
<li>Detection accurate up to 4 m​</li>
<li>Vision feedback in under 40 ms​</li>
</ul>
<h3>Key Responsibilities</h3>
<ul>
<li>
<p><strong>Software Architecture:</strong> Designed a centralized node structure using ROS2, where all but one node acts as Hardware Abstraction Layers, commanded by a TaskManager node. These HAL nodes were each responsible for a single hardware peripheral, such as a Detection node that uses the Realsense's Image feed, or the Arm Control node, which sends and receives information from the Dynamixels.</p>
</li>
<li>
<p><strong>Computer Vision Model:</strong> Created a perception stack that uses an Intel RealSense to perform efficient (3ms) bounding box detection in open environments</p>
</li>
<li>
<p><strong>CV Training Dataset Evaluation:</strong> Evaluated YOLOv11 performance across multiple datasets, ultimately deciding to use a custom dataset generated using Blender</p>
</li>
</ul>
<h3>Challenges Addressed</h3>
<ul>
<li>
<p><strong>Early into the project, the team experienced issues with personal accountability:</strong> To help lighten the load of responsibilities, daily meetings were scheduled for the mornings, before classes, to serve as a check-in and reminders of outstanding tasks (similar to scrum; however we did not enforce an amount of expected progress each day, moreso just taking note of roadblocks, enabling the rest of the team to help when needed).</p>
</li>
<li>
<p><strong>Team member consistently underperforming:</strong> One of our team members was consistently inflating his contributions during daily check-ins, and upon inspection the work did not hold up to standards. We organized a team meeting for the purpose of reviewing his work, and the conclusion was that scheduling cooperative work times would be helpful in keeping team members on-task while working in the lab, allowing underperforming members to work more efficiently.</p>
</li>
<li>
<p><strong>Portions of the code were developed with quick-and-dirty methods (connected to previous challenge), and then expanded solely with patchwork fixes, causing difficult and plentiful debugging:</strong> The core issue was that the team did not hold consistent code reviews or enforce standard coding practices. To fix the issues this caused, two members of the team (me and the one responsible for the code) sat down together to review the code. As problems were identified, I was able to come up with a simpler methodology, and wrote it in pseudocode, leaving him to finish the redesigned implementation. To finalize the new implementation, we performed a second peer review. While repeating the process elsewhere in the code resulted in a consistent codebase that held up to scrutiny by our advisors, many problems could have been avoided by holding to these standards from the beginning rather than starting with a quick-and-dirty solution.</p>
</li>
<li>
<p><strong>Crashed drone a week before final project deadline, burning out a motor:</strong> Following the crash, the team took a step back to fully understand what was left to complete for the project. As the only thing remaining that required the drone was the final integration test, we decided that our efforts were likely better spent creating our marketting resources (video + poster, both seen above), and while we did order new motors, they were not installed on the drone until after Project Presentation day. Focusing our attention on the outstanding tasks in other areas wound up being necessary, as we were working until the very last minute on the Project Report.</p>
</li>
</ul>
<h3>Project Specifications</h3>
<table>
<thead>
<tr>
<th>Purpose</th>
<th>Software Tools / Frameworks</th>
</tr>
</thead>
</table>
<hr>
<p>| Software Integration                   | ROS2 Humble (C++ &#x26; Python) |
| Software Logging + Visualization       | Rviz2                      |
| Computer Vision Model                  | YOLOv11                    |
| Computer Vision Library                | Ultralytics/PyTorch        |
| 3D Object Segmentation                 | PointCloud Library (PCL)   |
| Servo Motor Control                    | Dynamixel SDK              |
| Flight Controller Firmware             | PX4                        |
| Flight Controller to ROS Communication | uXRCE-DDS                  |
| Dataset Generation                     | Blender (bpy)              |
| CAD                                    | Solidworks                 |
| Kinematics Calculations                | MatLab                     |
| Version Control                        | Github                     |
| Kanban Boards                          | Github Projects            |
| Paper / Poster Making                  | Microsoft Office Suite     |
| Communication                          | Discord                    |
</p>
<p><strong>Software Design Patterns Used:</strong></p>
<ul>
<li>State Machine</li>
<li>Hardware-Abstraction Layers</li>
<li>Observer</li>
<li>Decorator</li>
<li>Singleton</li>
</ul>


Terrawarden Drone Cleanup

<h3>Inspiration</h3>
<p>This project was inspired by Michael Reeves's <em><a href="https://youtu.be/A_BlNA7bBxo?si=mZPmg3_MIDPBBQco">"Surgery Robot"</a></em>, which uses a VR camera to track a "surgeon's" hand in 3d space, and controls a gantry with a knife on it.</p>
<p>This, combined with Cam having access to a collection of 3D printer parts from a disassembled Taz, gave us a starting direction from which we found our vision for the Hand Machine.
What it does</p>
<p>Utilizing Google's Mediapipe ML/CV Framework, we track the player's hand with a camera+raspberry pi, and break it down into landmarks. <em><a href="https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker">(see here)</a></em> From there, we process this information, sending the centroid of the hand (relative to camera), as well as some gesture information (if the player has a closed fist) over serial to an Arduino, which controls the stepper motors and tracks the motion.</p>
<h3>How we built it</h3>
<p>Cam had parts for a LulzBot 3d printer, which we borrowed and reconfigured to create our claw machine. Kay brought in a Raspberry Pi 4b to run the CV model on. We mostly worked in the Makerspace, so we utilized our access to the laser cutters in the Prototyping Lab to create the parts we needed.</p>
<h3>Challenges we ran into</h3>
<p>Our team knew we were quite busy on Sunday, and didn't want to be forced into pulling an all-nighter to finish this project. As such, everything had to be constructed with parts we had lying around (plus materials purchased through the Makerspace for the laser cutter). This project would not have been possible without the 3D Printer hardware that we had access to.</p>
<p>The camera we use has a very heavy fish-eye lens. Its field of view is massive, and that made controlling the machine with your hand quite tedious (because the play area was effectively a dome). We solved this by cropping the camera's field of view (so the FoV is still a dome, but you don't need to get nearly as close to the camera to go the the extremes)</p>
<h3>Accomplishments that we're proud of</h3>
<p>Setting up the computer vision pipeline went really smooth. Google's Mediapipe framework had a pretrained model for hand+landmark detection, so the code was "simple" to write. From there, the process layer running on the Raspberry Pi wasn't too terrible to write (we iterate through the hand landmarks to identify which of the players fingers are pointing upwards, which we use to determine if their hand is closed).</p>
<p>In the meantime, we were working on the stepper motor control. Our stepper drivers have speed control, and we implemented an acceleration layer in order to have smooth position control. This is controlled through an Arduino clone, which receives control commands through Serial from the Raspberry Pi.</p>
<h3>What we learned</h3>
<p>From this project, we learned a lot about the design of gantries, how to make sure they are squared and level, and how to tune and optimize their performance. We also learned about camera lens distortion, and how smaller cameras utilize them to get a much larger field of view, as well as how to apply small corrective measures to improve the quality of life of the end user.</p>
<h3>What's next for Hand Machine</h3>
<p>Future plans for the project include improving the vision system to allow for full 3D control of the gantry, likely by using the size of the hand as a depth estimator.</p>
<p>We also would like to add more quality of life / fun features, such as a laser indicator for the gantry, and a buzzer or speaker to play music.</p>


The Hand Machine

<h2>Project Description</h2>
<p>This project involved the creation of a convolutional neural network with a Unet architecture for the identification of drone racing gates. The network was trained and tested on a dataset artificially generated from Blender and augmented using Python’s torchvision.transforms library. Additionally, the network reached an accuracy of 95% on the testing sample and demonstrated success at tracking gates in a sample video that was not a part of the dataset.</p>
<h2>Methodology</h2>
<p>Our team used a rendering software called Blender to generate 3D scenes. Blender contains a built-in Python interpreter and API, allowing users to write scripts to manipulate and export scenes. We set up a scene with multiple windows on a black background. Our image generation script would randomly move windows, import backgrounds, adjust lighting, and change the camera position within the scene five thousand times, exporting the scene to an image after each iteration.</p>
<p>We created an image augmentation script to import the images and apply various transformations onto them, expanding our dataset size to fifty thousand images. Transformations included rotation, resizing, blurring, adding noise, adding color jitter, inversion, and simulating the image being put under water.</p>
<p>We decided to use the U-Net architecture for our model, which consists of an encoder to extract image features and a decoder to create a mask out of these features. In our specific case, the model should predict the location of the windows in an image. The output labels were a mask of the original image, where the window targets are white and the rest of the image is black, as seen above.</p>
<p>With U-Net's encoder-decoder architecture, we have four encoder steps, four decoder steps, and a bottom step linking the two. These steps consist of a small pattern that repeats twice, as well as either an Upsample or MaxPool2D step. This pattern consists of convolution layer (which was padded so that it did not change the size of the image), a batch normalization layer to normalize the inputs, and a ReLU activation layer (which converts all negative input values to zero but keeps other input values the same). The batch normalization layers prevent the input values from being too big, preventing the output from growing to infinity.</p>
<p>Each encoder layer splits up the image into smaller pieces, the first of which are usually called feature layers. Each encoder layer is the pattern we described earlier, twice, followed by a MaxPool2D step. The MaxPool2D layer is what actually decreases the image size, using (in our case), a kernel size of 5, a stride of 2, and a padding of 2. During our encoder steps, we gradually decrease the size of our kernel, starting at 11, then decreasing to 7, 5, and finally 3, where it remains for the rest of our Conv2d layers.</p>
<p>Each decoder layer combines these features back into a larger image. Each decoder step is an Upsample layer, which actually does the combining, followed by the same pattern mentioned twice already. Separating the encoder+decoder layers is a bottom step, which is just the same pattern we've mentioned thrice.</p>
<p>Most U-Net architectures contain skip connections, which preserve the outputs of previous encoder layers and add them to the corresponding decoder layers, thereby preventing the gradients from reaching zero and freezing progress within the neural network. We initially wrote our network intending to add skip connections later, but when we trained our network, we found that its accuracy was sufficient. Given the amount of time to retrain our models, we decided to forgo the skip connections and stick with our current structure.</p>
<p>In training this model, we utilized Binary Cross Entropy Loss (BCELoss), which uses the log of the model output, along with the label, to determine the loss. BCELoss is a common solution for binary classification, which matches our use case. For training, we utilized ADAM with a learning rate of 1e^-4.</p>
<p>We trained with batches of eight images for 24 hours, totaling around 433,000 batches and 69 epochs. At around 100,000 batches and 15 epochs, our model settled around a 5% training loss, only improving to 4% in subsequent batches. However, at this step we realized that within our dataset, we made the windows too small, such that it did not accurately reflect the video we were to test against. We needed to regenerate our 5k renders from Blender with a larger scale for the windows, then retrain for another 8 hours, totalling 140,000 batches and 21 epochs. We processed the final video using the model after 21 epochs, at a loss of 12.0% (or 88.0% accuracy). We also tested with partially trained models, using epoch 3 (38.5% loss) and epoch 13 (12.8% loss) for comparison, but the later model was either equal or better on every frame, so we kept the newer model. A video of this comparison can be seen above.</p>
<p>The model's training loss progression can be seen above. At the beginning, the model improves drastically, but once it hits about 40k batches (only 6 epochs), the model barely improves at all. By batch 100k (epoch 15), the validation loss had only improved 9%, from 23% loss at batch 40k to a 14% loss at 100k.</p>
<p>After retraining our model to perform better, it did quite well on the final video. However, we once again did not account for exactly how large the window gets in the video, and it once again was not reflected properly in the dataset. As such, that part of the video is lacking in clarity compared to the rest.</p>
<h2>Reflection</h2>
<p>Overall, this process taught us valuable skills in image generation, dataset augmentation, and implementing more sophisticated machine learning architectures. One mistake we made that we would do differently next time, was not checking the video before generating our dataset. In the final video that we tested our dataset against, the windows were much closer, took up much more of the screen, and were sometimes cut off by the edge of the frame. In contrast, the dataset we trained our algorithm on had much smaller windows, often showcased these windows at a much sharper angle, and rarely had windows that were cut off by the edge of the frame. If we had accounted for these attributes in our dataset, we may have had more success when testing it. Despite this, we were still able to achieve an accuracy of 88.0% for our test dataset and adequately track the frames in the test video.</p>


Semantic Segmentation Using Deep Learning

<h1>The Fastest Team</h1>
<p>Midway through my Junior Year at WPI, I took Unified Robotics IV: Navigation. This course focus on concepts very commonly found in robotics: localization, path planning, and navigation. To do so, we utilized the Turtlebot3 robot, along with its LiDAR scanner. Our goal was to program the robot using the Robotic Operating System (ROS) to autonomously map an environment, return to the starting point, and then navigate to any point we command within the map. Afterwards, we were to perform the "kidnapping problem", where the robot is then placed in an unknown location within the map, and told to drive to some other location. The robot is to determine where it is on the map, and then manage to navigate to this new target. The team whose robot completed these steps the fastest were granted extra credit.</p>
<p>Our team consisted of three insanely talented and dedicated members who were a pleasure to work with. The experience was both challenging and enjoyable, as we not only completed the assigned task but also used the remaining time to experiment with additional features, such as localizing without any physical movement. For every little bit we wrote custom ROS launch files. Additionally, two of us opted to run everything on Windows Subsystem for Linux (WSL) rather than Ubuntu, purely for the fun of the challenge and to avoid dual-booting our personal computers.</p>
<h2>The Process</h2>
<p>The first step to this process is the mapping. Using the LiDAR, we were able to take note of the walls could see. Then, we looked for the frontier, the boundary between what we could and couldn't see. We used OpenCV to find the centroid of this boundary, and set that as the robot's target position. We utilized the A* algorithm to find the shortest path to this point, using a modified heuristic to encourage the robot to stay away from the walls as much as possible, as well as discourage turning as much as possible. Path following was then executed using the Pure Pursuit algorithm, which tracks a position some distance ahead of the robot on the path, allowing the robot to preempt turns and make adjustments accordingly. The robot then repeats this procedure until no more accessible frontiers can be found. Videos of this, both of the robot, and of what the robot sees (shown using ROS Visualization, or RViz) can be seen above.</p>
<p>Once the robot completes the mapping procedure is complete, we use A* once more to find our way back to the position we started in. Then, we pick up the robot and move it elsewhere within the maze. The robot spins in place, collecting data until it is confident that it knows where on the map it is, then waits for a target position to be assigned. It pathfinds back, stopping occasionally to confirm its position and the optimality of its path. A video of this, very nicely edited by one of my teammates, can also be found above.</p>


Robot Navigation - SLAM and AMCL

<h2>Description</h2>
<p>For my Humanities Practicum at WPI, I elected to take a class called Musical Robotics. The goal of the class was to develop an innovative musical robot, from theory to implementation, in just seven weeks.</p>
<p>Based on a children’s toy from the 80s, the Whirly Tune is a customizable ensemble of corrugated tubes mounted to motors that each produce a unique pitch when spun. Controlled by three separate Raspberry Pi Picos, our design is highly modular and customizable. With a timbre that is both eerie and ethereal Whirly Tune has the potential to be the perfect accompaniment to many performances.</p>
<p>Whirly Tunes is controlled by a serial connection to Ableton via a patch written in Max MSP. This means it can easily perform MIDI pieces or be played in real time via a MIDI keyboard.</p>


Whirly Tunes

<h2>Description</h2>
<p>During D-term of Sophmore Year, I took WPI's CS 3733 Software Engineering class. The class has a notorious reputation as one of the hardest classes at the school, due to the large team size and its very short iteration cycles. Through two prototypes and four iterations, each developed during week-long agile sprints, I was forced to learn how to manage a team of 11 developers as the team's Lead Software Engineer. Our resulting app used Java, with JavaFX for the UI/UX interactions, and SQL (including a AWS-hosted server) on the back end.</p>
<p>The app we made was a program to be used at various kiosks around the hospital, to be used passively as signage, more actively for pathfinding around the hospital, or by staff to submit work orders or food delivery requests. To the right, you can see a trailer that we made for our app, which is used as a screensaver after a period of inactivity, which showcases some of the features that we implemented.</p>
<h2>Our Approach</h2>
<p>To manage a team of 11 people, it was important we chose the right tools for the job. We used Github for software version control, used Jira for User Stories and a Kanban board, and used Figma for UI Mockups before they were created in our Java Application.</p>
<p>One of the most important takeaways from this class was the utility of Design Patterns. The most prevalent ones for this project were ORMs, DAOs, and the Facade patterns. These three together managed all interactions with the SQL Database.  The ORM objects mapped to the individual rows of a given table, the DAOs managed all communication with those tables, and the Facade hid all of those interactions behind a singular Singleton so that the rest of the program did not need to know anything about the database beyond what it needed to give/get from it.</p>
<h2>Lesson Learned</h2>
<p>The class being so work intensive meant that there was a lot to learn. For sure the hardest part of the class was managing splitting all the work between the 11 of us, but once we got comfortable with the pacing of the Sprints, we stopped having to grind on the last day before each submission.</p>


Brigham and Women's Hospital Application

<h2>Description</h2>
<p>During C-term of Junior Year, I took an experimental course: RBE 470x: AI for Robotics. The class was a new course, which ended up being a cross between WPI's Intro to AI and Machine Learning courses. In this class, we covered everything from Optimized Graph Search to Markov Decision Processes, to Reinforcement Learning. The centerpoint of this class was this python implementation of Bomberman, which the professor provided. Our two major projects for the term were to implement some of the algorithms we discussed in lecture within this environment, to create our Bomberman AI.</p>
<h2>Bomberman AI Part 1: Expectiminimax</h2>
<p>This part of the project had 5 phases. Variant 1 was trivial, just get your code to walk to the end. The Variant 2 added a "stupid" monster, who moves randomly each turn. Variant3 changed the "stupid" monster to a "self-preserving" monster, who moves in a straight line, bouncing off walls at random angles, and will attack if the player is within 1 unit of the monster. Variant 4 once again changed the monster type, this time to an "aggressive" monster, who behaves similar to the "self-preserving" monster will move towards the player if they are within 2 units, not just 1. This gradually increasing difficult defined how we approached this problem.</p>
<p>We decided on an Expecti-minimax approach, which allows us to factor in the randomness of the monsters, while still beinng optimal when the self-preserving or aggressive monsters are acting deterministicly. A core feature of this approach is the ability to evaluate states using a custom heuristic, which will simplify a complex world down to a numerical rating.</p>
<h3>Our Heuristic</h3>
<p>The first thing we wanted to do was ensure that our player knew to approach the exit. Rather than calculate A* to find an optimal path every turn, which would explode in time complexity when we run a tree search, we start the program by calculating a wavefront, starting from the exit. Our heuristic starts by grabbing the distance from our saved wavefront. Then we punish it for being too close to the monster (but only if the monster is closer to the exit than we are), and reward it for having more options for moving around (as to discourage the player from hugging the walls). We also factor in the "events" that the game supplies, which tells us if the player dies, wins, or killed a monster.</p>
<h2>Bomberman AI Part 2: Approximate Q-Learning</h2>
<p>The next part of the project was not nearly as simple. We were asked to implement Reinforcement Learning, either using by use of a library or manually. We chose a manual implementation, since I thought it would be just as difficult to use PyTorch or another similar library as it would be to do it ourselves. Also, in my own (very biased) opinion, it would be more fun.</p>
<p>Our Reinforcement Learning Algorithm of choice was <strong>Q-Learning</strong>, specifically Approximate Q-Learning. Approximate Q-learning is a simplified form of reinforcement learning where the utility of a state is a linear combination of features defined within the state. These features are traditionally normalized between 0 and 1, and should be representative of all the information the model needs in order to survive. The coefficients (weights) for each of these features are iterated on mathematically throughout training.</p>


Bomberman AI

<h2>Abstract</h2>
<p>The goal of this project was to provide recommendations to the Worcestershire Local Nature Recovery Strategies (LNRS) Preparation Team for how access and perceptions of local green spaces can be enhanced alongside habitat creation and restoration. At events with the sponsors, infographic boards were used to gather direct data from residents in two communities of Worcestershire County (Westlands Estate and Warndon Villages) about their opinions and perceptions about the green spaces near them. An online survey was also used to gather related data based on certain demographics, and several local project groups were contacted and interviewed on how they would improve green spaces and access to them. It was observed and documented that while perceptions of green space quality and accessibility were similar between the two communities, there were more observable issues with the Westlands’ infrastructure and knowledge of green spaces. It was determined that there were many educational, infrastructural and engagement-based initiatives that the LNRS Preparation Team can undertake to help both communities improve their access to local green spaces and residents’ knowledge of the green spaces.</p>
<p><strong>This project was undertaken as part of WPI's Interactive Qualifying Project Program. Please view the attached link for more information.</strong></p>


Recommendations for Local Nature Recovery Strategies (LNRS) in Worcestershire, UK

<h2>Description</h2>
<p>During my junior year of college, I took Robotics Engineering 3001: Manipulation. This is a course that focuses on the controls of multi-degree of freedom manupulators and the related mathematics.</p>
<p>Our team of 3 was responsible for deriving the forward and inverse kinematics for our 4dof Arm, as well as implementing Velocity and Trajectory-based controls in both Joint Space and Task Space, over the course of the seven-week term.
We utilized the Denavit-Hartenberg Convention for calculating the Forward Kinematics.</p>
<p>The full math and writeup for our report can be found in the linked report. Please note that the github link found in the report no longer functions. (It was originally included for the grader's convenience)</p>


Kevin Siegall

Roboticist • Software Engineer

All Projects

Kevin Siegall