Technologies: Arduino Yun, NodeJS, JavaScript, PID-control - communication: Arduino/tangible-programming-board (Bluetooth), Arduino/server (TCP: HTTP-request), server/web-client (TCP: SocketIO)

I developed the BRICKO 2.0 Educational Robotics system as part of my master thesis.

A slide

Technologies: Unity, Meta Quest 2 VR, microcontrollers (ESP32/Arduino/Pico Pi W), Blender, 3D printing - communication: microcontrollers/Unity/Quest 2 (UDP)

Imagine being able to build your own LEGO models and then being able to actually touch and use these in VR, to interact and control fantastic vehicles and worlds!

With the new system for creating interactive tangible VR experiences that I have developed, you'll be able to do just that.

A short demonstration can be viewed below, followed with a brief description of the system and the process involved in setting it up.

-- Demonstration --
LEGO VR - custom build LEGO steering wheel with speeder and brake triggers.


Robotics VR - custom build robotics arm build using premade model-components from the developed system, featuring three interactions points.

-- Gallery --

-- The System (in short) --

The system works by automatically handling the synchronization between real-world objects and their virtual counterparts, including the manipulation of their interactive points - the real-world manipulation is then updated virtually through sensor feedback send wirelessly from a microcontroller to the Unity application using UDP.

All the user need to do is create a 3D and real-life model*, implement the desired sensors (potentiometers/switches), then define its static and dynamic parts in Unity and register it as a tangible VR object.

The system then automatically handles and auto-generates the necessary microcontroller code, communication protocol, synchronization etc.

*The system also provides a range of pre-made models (static and dynamic) that users are free to use to construct their own objects from, in a similar fashion to LEGO.

-- The Process --

Model - Create the model:
Build your model in real-life, then replicate it in Mecabricks or STUD.IO*

Then open the model in blender and reset the origin point for each moveable part, thus defining their interaction points.

*Alternatively 3D model any type of object and then 3D print it.

Microcontroller(s) - setup microcontroller(s):
Select your type of microcontroller(s) (ESP32, Arduino, or Pico Pi W) and define your sensors, and WiFi connection details - Unity will then auto-generate the necessary code for your microcontroller*.

Attach your sensors (potentiometers/switches) to your microcontroller and your models interaction points.

*To easen the user-interaction with the system, I developed a Unity plugin which auto-generates the neccessary microcontroller code.

Unity - setup game world:
Select the developed OriginSynchronization prefab-object and drag the individual parts of your custom models on top of it*.

*The system supports an infinite number synchronizable objects, enabling users rich posibilities to populate their gameworlds with interactive objects

Unity - setup interaction:
Attach the developed TangibleVRInteraction component to each of the interactive parts of your model and drag these onto the UDPDataReceiver object.

Unity - setup synchronization:
Drag your your model onto the TangibleVRSpawner object.

Playing - your game:
Press play and put the VR headset on.

Use the analog-stick on the right VR-controller to select between your synchronizable objects and spawn these using the trigger button.

Double tab the controller against a surface to enable hand-tracking.

You can now touch and interact with your own custom models in fantastic VR-worlds.

A few assorted projects.



LEGO House - Fish Creator:

To demonstrate the process of reading pixel colors from images and using Unity to generate 3D models, I created a basic example for a couple of students. The example was inspired by the Fish Creator installation at LEGO House.

Blender - A random room:

A random room I modelled in Blender af few years ago.

MAARS - Multi-Agent Autonomous Robot Systems:

I collaborated with a friend to program three sets of LEGO Mindstorms NXT robots using RobotC. Their objective was to sweep an area, map its colors, and search for a specific treasure square. The robots were equipped with GPS modules and color sensors for navigation, and they communicated via Bluetooth.

Since the GPS modules had a three-meter accuracy limitation, we had to carry the robots around outside for testing. However, a scaled-down version of the area allowed us to test the sweeping algorithm and communication indoors, relying solely on the color-sensor.

Sokoban Solver:

As part of an assignment in an AI for embedded robotics course, I developed an algorithm to solve Sokoban courses, outputting the necessary instructions to be performed by a Mindstorms NXT robot (programmed in Bricx Command Center).

Drawing inspiration from key-frames in hand-drawn animation, I implemented a similar system that could generate the missing steps between two key-frames. This modification significantly improved the average runtime for computing a solution. The standard breadth-first search used in the course took 10-15 minutes, while my modified version with the key-frame animation algorithm overlay applied reduced the runtime to just 3-5 seconds.

2D Game Prototyping

Here are a few images showcasing various 2D point-and-click games and physical prototyping of mechanics. Some of these were developed alone, while others were created together with friends.

HEXACore:

During an Experts in Teams course, I collaborated with five other students from diverse engineering disciplines. Our projects focused on developing a modular robotics base for mobile robotics, equipable with a wide range of tools.

Below are some sketches I created for a poster presentation showcasing our project.

Technologies: micro:bit, breadboards, sensors and actuators - communication: micro:bit/computer (Serial), micro:bit/micro:bit (Radio)

Since Teknologiskolen opened in 2015, we have had very few girls attending our classes, usually limited to one or two girls per season. To address this, I collaborated with attending girls over the years, experimenting with different project contexts and construction materials to make the classes more engaging for them.

In 2020, I applied all the insights I had gathered and started a girls-only team at the annual summer camp, using a distinct approach and setup compared to our usual LEGO Mindstorms and instruction-heavy Arduino projects. The focus shifted to instruction-less projects, providing ample opportunities for personalization and creativity, while utilizing familiar materials like canvases, paint, cardboard, wood, glue guns, scissors, crayons, markers, popsicle sticks, magnets, pipe cleaners, steel wire, feathers, cotton, sequins, etc.

The micro:bit platform was incorporated for a closer alignment with school curricula, empowering the girls to apply their learning in future classes.

The girls-only team quickly gained popularity and was fully booked, becoming an instant success. Since then, I have taught a girls-only team each season at both our annual summer camp and our weekly classes.

Although we have introduced additional projects over the years, the initial projects of Interactive Art Installations and Smart Homes/Doll Houses remain the most favored among the girls attending the camp.

A video of an example project I made can be seen below, followed by a series of pictures from this years summer camp.

Technologies: 3D Modelling and Printing, Laser Cutting, Fritzing

During my PhD, I developed a non-intrusive DIY re-design and upgrade for existing breadboards, along with a re-designed of circuit diagrams. These were aimed at simplifying the learning of electronics and microcontroller technology in K12 education.

The re-design combined strengths from standard circuit diagrams (simplicity, non-location-specific) and Fritzing diagrams (contextualized visuals), while bridging the implementation gap to breadboards and addressing their respective weaknesses.

Breadboards: I created a non-intrusive 3D-printable/laser-cuttable overlay, highlighting internal connections.

Circuit Diagrams: I incorporated visual cues from the breadboard overlay, while avoiding the location-specific issues of Fritzing diagrams.

After several iterations and years of testing, the results were very positive. Pupils found it easier to understand and work with breadboards, made fewer connection errors, and improved their circuit debugging skills. The non-location specific circuit diagram redesign enabled them to combine multiple circuits on a single breadboard, while the simplicity enhanced their understanding of how electricity flow.

To assist others in using the new system, I developed a Fritzing compatible library (including component graphics) for drawing circuit diagrams of the new design, as well as an electronics compendium for interfacing sensors and actuators with the micro:bit microcontroller platform.

Images of the breadboard overlays, redesigned circuit diagrams, project examples, and compendium pages, can be seen below.

Technologies: JavaScript, NodeJS, NoSQL, Fritzing

I have developed a digital compendium for electronics, allowing users to choose their microcontroller platform, connector/shield/peripheral, programming language, and circuit diagram type. It supports the primary school sector and is currently being implemented as a part of www.teknologiundervisning.dk

The hardware components list automatically updates based on the user's selection criteria. When viewing a component guide, users can choose how the programming language is visualized: as an image, an embedded code editor, or text. They can also switch between supported microcontroller platforms, languages, and diagram types.

For example, a primary teacher using the micro:bit platform to teach pupils transitioning from visual to textual programming can select micro:bit (platform), MakeCode, and JavaScript (language). The component list will only show components with guides for the micro:bit platform and both MakeCode and JavaScript. These guides can be printed and used in class to teach similarities between the languages and how to use a component with either language.

Internally, the compendium data is stored in a tree-structure, allowing multiple combinations to share resources like descriptions, images, code examples, and diagrams.

Images of the compendium can be seen below.

Technologies: Arduino Lilypad, MP3, 3D modelling and printing - communication: Lilypad/Lilypad (I2C)

Together with two friends, I augmented a Buzz Lightyear playsuit for kids, bringing Buzz Lightyears functionalities to live.

We incorporated five functions based on the original suit's button layout: one for Buzz's famous quotes, another for activating an invincibility force field, two for shooting (proton missiles or pulse bursts), and one for selecting between the shooting modes. These functions included light and sound effects. The shield could last for 10 seconds before needing to recharge, and the kids had to time the charging and release correctly to shoot the missiles successfully; otherwise, it would fizzle out.

We refined the suit through multiple iterations of playtesting at kindergartens, and images of the prototype can be seen below.

Technologies: Arduino Uno, Stud.IO, 3D modelling and printing

In 2019, I organized and taught two weeklong science camps on robotics and automation for 7th-9th grade school pupils in Kiel, Germany.

The project I had developed for the camps challenged the pupils to act as engineers, tasked with developing robotics for automating a coffee bean central. They could choose between creating an automatic sorter to classify coffee beans by quality (color-sorting) or developing automatic bean transporters to carry the sorted beans to their respective storages inside the warehouse (line-following).

The project utilized the Arduino microcontroller and LEGO Technic as the construction platform. To facilitate integration, I designed various 3D-printable modules bridging the gap between electronics and LEGO. Additionally, I developed LEGO manuals for constructing a base-model, offering a foundation for further expansion.

The LEGO manuals can be downloaded below.

Bean Sorter (rotator):

Bean Sorter (conveyor belt):

Bean Transporter (cyan and magenta indicates 3D modules):


Videos and images from the camps, projects and materials, can be seen below.

Technologies: Arduino Yun, NodeJS, JavaScript, PID-control, Laser Cutting, PCB development - communication: Arduino/tangible-programming-board (Bluetooth), Arduino/server (TCP: HTTP-request), server/web-client (TCP: SocketIO)

As a part of my master's thesis, I created the BRICKO 2.0 Educational Robotics system. This system shared similarities with LEGO Mindstorms, featuring a controller module that could be expanded with various modules, including sensors and actuators.

The primary aim behind BRICKO 2.0 was to facilitate the seamless integration of Educational Robotics into primary school classes. Additionally, the system was designed to be a sustainable investment by catering to a wider range of users across different age groups.

Integration into primary school classes:

After closely working with local primary schools, teachers, and students, I discovered that despite having LEGO Mindstorms systems provided by the municipality, they were rarely used. The reasons included time constraints in unpacking and setting up the system during regular school classes, as well as the need for GUI installations on students' computers.

To overcome these challenges, I created the BRICKO 2.0 system. It is modular with a velcro-based design, requires no additional programs, is OS-independent, and operates directly from any web browser.

Age-range of users:

BRICKO 2.0 catered to a wider age-range of pupils with both tangible and visual block-based programming. The tangible language remained consistent with BRICKO 1.0, while the visual language was developed specifically for BRICKO 2.0.

Multi-language support: To support students in "indskolingen" (1st to 3rd grade) and "mellemtrinnet" (4th to 6th grade), a multi-language approach was implemented. This allowed schools to invest in a single Educational Robotics system, reducing costs and enabling more widespread implementation.

While there were plans to introduce a textual programming language for "udskolingen" (7th to 9th grade), it was not completed due to time constraints.

Vertical coding: BRICKO 2.0 adopted a vertical coding style similar to traditional textual programming languages, making the tangible and visual languages more familiar and facilitating an easier transition.

LEGO Spike:

Two years later, LEGO adopted similar approaches for the Spike system, implementing multi-language support, vertical coding, and introducing new elements for easier construction.

An overview of the BRICKO 2.0 system:

Modules: Modules: The controller supported up to three DC or servomotors and three additional modules, including a buzzer, color, distance, pressure, and rotation sensors.

It featured a slide switch to toggle between tangible and visual programming modes, while an RGB-LED indicated the current status and connection. Similar to BRICKO 1.0, a LEGO brick could be placed on top to determine its personality mode.

GUI: The GUI, accessible via any web browser, featured a drag-and-drop block-based language with support for nested loops, conditionals, and functions.

It consisted of four dynamically expanding coding rows: one for the main program and three for functions.

Conditional blocks were specifically designed for use with the developed modules, incorporating interval sliders as a simpler alternative to traditional "if (value > something && value < something)" statements.

The GUI provided real-time feedback by highlighting the corresponding blocks during command execution or conditional evaluation. This visual feedback allowed users to follow the system's logic, making it easier to identify potential errors and debug their code.

At the bottom left, a grid displayed the connected modules and their current states.

Internal Hardware: The controller module's internal hardware was based on an Arduino Yun, extended with custom-made shields stacked on top. Additionally, the DC motor modules were equipped with tachometers for PID control.

Usage examples:

Alarm systems and train track shifting are simple scenarios demonstrating simple input-output systems.

User tests:

During development, I consistently conducted user tests to assess the response of previous and new users to implemented changes and the system.

Technologies: Arduino Nano, P-control, Laser Cutting, 3D modelling and printing - communication: Arduino/tangible-programming-board (Bluetooth), Arduino/HC-05/06 (UART), Arduino/SD-Card-Reader (UART)

BRICKO is a LEGO-based system features a driveable robot with a tangible programming interface. I co-developed it with a friend during our bachelor's thesis (version 0.5). Afterwards I continued the developedment together with three other friends (version 1.0), complete with a 4th grade curricula and a series of video tutorials.

Robot & Language: The robot can be set to different personalities (robot, dog, police car), influencing its movement length and sounds during commands.

The tangible language supports commands such as drive-forward/backward, turn-left/right, make a sound, loops (2/3/4/7/∞/end, also nested), and functions (also nested and recursive).

Programming Board: Programs are built on the programming board and can be transferred via Bluetooth to the robot with a press of "play," allowing for pausing and stopping. LED indicators show the current command and syntax errors (unclosed loops).

Compendia: Designed for 4th-grade mathematics and Computational Thinking, the compendia supports both low-floor and high-ceiling tasks, offering successful experiences for all students while challenging experienced users. It included three types of tasks:

1) Drawing the expected outcome of a given program, equivalent to first-order equations, to train pupils in decomposition and algorithmic thinking. For example, 20 * x = 80 can be represented as drive-forward (20cm) repeated 4 times = 80cm.

2) Coding the robot's behavior to match a given drawing, requiring pupils to fulfill specific requirements and develop skills in pattern recognition and algorithmic thinking. For example, coding a square: loop-4x, drive-forward, turn-left, loop-end.

3) Contextual tasks involving catching thieves on floormats representing city streets (police car) and park areas (police dog). These tasks encourage abstraction and help students realize that the floormats consist of nodes and vertices.

Video Tutorials: The tutorials were designed to work with and without sound, suitable for use both inside and outside the classroom.

The Development Process:

A series of images and videos showcasing the development process from inception to the finished product, can be seen below.

BRICKO 0.5 - Images

BRICKO 1.0 - Images

BRICKO 1.0 - Videos

The driving distances and sounds of the personalities

The pause function and LED/sound-feedback

Pupils working with the compendia

Technologies: Meta Quest 2, Unity, Blender, Arduino WiFi Rev2, NodeJS - communication: Arduino/server (UDP) Unity/server (TCP: SocketIO)

I wanted to experiment with blending the physical and virtual world - enabling physical controllers to become tangible components of virtual steampunk airships, gigantic mecha cockpits, escape rooms, POD-racers etc.

Imaging designing and building your own LEGO models and for these to become interactive peripherals. Peripherals which you can then physically interact with and manipulate in the real world, to control your own VR spaceship while being deeply immersed in exploring the final frontiers of the universe.

The idea is to create a DIY modular IoT controller system for the Quest 2 headset and Unity, enabling others to combine functionalities for their projects. Modules will have a default base-template to synchronize their position with the headset and a single functionality users can expand upon, including sensors (rotation, pressure) and actuators (e.g., servo motors).

Users can then download the 3D model of the base-template, design attachable components (e.g., a leverarm), and integrate them into both the physical and virtual models. After synchronization, the controller exists in both worlds, allowing users to explore fun and creative ideas.

Watch a short video below to see the overlap between the physical and virtual realm.

Click on the images to read a short description of the different parts of the project.

Technologies: JavaScript - communication: micro:bit/PCA9685 (I2C)

I wrote the library for (and named) the FireFly, a micro:bit extension board created by one of my colleagues. This board enables the micro:bit to connect to a breadboard and control up to four DC motors and four servo motors. The library interfaces with the board's built-in PCA9685 and L293D chips.

Currently, the FireFly is sold out at Teknologihuset, and a version 2.0 is on its way.

Images from the library can be seen below.

Technologies (a): NodeJS, Electron (framework), micro:bit - communication: micro:bit/computer (Serial)

Technologies (b): Swift (iOS), micro:bit - communication: micro:bit/iPad (Bluetooth)

a: Before Microsoft implemented a serial monitor in MakeCode, users had to depend on third-party monitors or the built-in 5x5 LED-matrix. These third-party monitors were often complex and designed for engineers and enthusiasts, making it unnecessarily complicated and frustrating for teachers and pupils to work with the micro:bit platform.

To address this, I developed a user-friendly monitor with data logging functionality for the schools we collaborated with and other micro:bit users. Since then Microsoft has introduced their own monitor in MakeCode, and the rest is history.

Images of the monitor can be seen below.



b: To provide users with a convenient way to monitor and log data from the micro:bit without the need for a second micro:bit unit connected to a computer through radio communication, I developed a new application. This time, the application was designed for iOS and iPads.

The iOS application allowed users to utilize Bluetooth to monitor and log the state of the micro:bit's built-in and external sensors. The logged data could be sent via email to the user's account as a .csv file, making it compatible with Google Sheets or Excel. Unfortunately, due to other commitments, I had to discontinue the development before finalizing it.

Watch a short video below demonstrating how to download a program to the micro:bit, monitor sensor data, and send it by email.

Technologies: micro:bit

In 2022, I traveled to Namibia for a two-week collaboration with the University of Windhoek (Namibia) and the University of Turku (Finland). During my visit, I conducted two workshops—one for Namibian engineering students and another for Namibian primary school pupils. The pupils and students were some of the most open, friendly, and inspiring people I have ever met.

University Workshop: At the university, I delivered a lecture on teaching robotics and Computational Thinking using Educational Robotics, followed by a hands-on workshop.

The focus was to equip university students with insights on introducing robotics to primary school pupils, aiming to build a community similar to Teknologiskolen in Odense.

Primary School: In collaboration with the local school in Nakayala it was organized for the engineering students to visit and help conducting a robotics workshop for the pupils.

The concept was well-received by both students and pupils, as they enthusiastically worked on developing automatic plant-watering systems.

Technologies: Unity, Meta Quest 2/3 VR, microcontrollers (ESP32), Blender, 3D printing - communication: microcontrollers/Unity/Quest (UDP)

I am currently working on a series of wireless interactive LEGO units.

The system:

Brains: The main idea is to have a series of "brain" units with to dials and an ID field.

The brain units can be attached to different sensors and actuators to control these, using two dials.

Sensors: The dials controls the sensors thresholds.

If the sensor value falls within the threshold, it emits a positive respons for the dial (A or B), to all units with the same ID.

Actuators: The dials controls the actuators behaviour.

If it receives a positive response (A or B) the actuator initiates and act accordingly to the dial.

Use Cases - Without a Computer:

The ID bricks and dials is all you need

Smart Home Christmas Ornaments
Set a lightsensor to emit A when light, and B when dark.

Set a neopixel to turn off when A, and light up when B.

Now your room christmas decorations will shine bright when it is dark outside, and turn of when it becomes day.

The Mysterious Cat
Set a distance sensor to emit A when something is close by, and B when there is nothing in front of it.

Set a DC motor move forward off when A, and stop when B.

Now your LEGO cat will tease you, by moving away everytime you try to get close to it. Can you sneak up on it?

RC MechaRobot
Set a button to emit A when pressed and B when not pressed. Set a rotation sensor to emit A when turned left and B when turned right.

Set a DC Motor to move when A and stop when B. Set a Servo Motor to turn left when A and right when B.

Your mecha robot is ready for action, can you navigate all the obstacles and save the day?


Use Cases - Programming with a Computer:

The units can be used wirelessly together with a virtual programming language, with sensors acting as inputs and actuators as outputs.

In this way, the units will be similar to LEGO Spike/Mindstorms, but each unit will function independently and wireless.

Mood Frame Friends
Write a small program, that send your rotation sensor to your friends neopixel unit and vice versa.

Do you need a hugh when coming to school tomorrow or are you in a happy mood? Turn your rotation sensor to reflect your mood and let your friend now how you feel.


Use Cases - Gaming with a Computer:

Just like Makey Makey you can use the units to control your favorite games.

Super Mario
Use your button or other units to create your own controller for Super Mario and go on daring adventures.


Use Cases - Gaming with VR:

Why stop at your floor or table, why not jump right into fantastic VR worlds?

Space Explorers!
Build your own spaceship controllers using your LEGO bricks and wirelss units, in both reallife and 3D using Mecabricks or STUD.IO

The sensor units will transmit your interactions, allowing you to not just see fantastic worlds, but feel them too!