I’ve used Twilio for a while for programmatically sending and receiving SMS messages. There’s also a visual editor called Studio that can be used to make call and message flows:
It can be connected to Twilio Autopilot to make AI-powered bots. Tasks are trained with sample phrases. These sample phrases are variations on what would be said to trigger an action e.g. ‘Call reception,’ ‘Front desk,’ ‘Talk to a human.’
An example that comes to mind, is making a call handling system for an office. Rather than a voice menu that details each option followed by a number, the caller could simply say who they wanted to talk to or what their request was about, and the system would handle it. This is far more respectful of the caller’s time compared to having them listen to a long list of choices.
It works with SMS and voice calls, and seems a good way to build an IVR (Interactive Voice Menu) system. TwiML can be used for more complicated tasks, while still using Studio/Autopilot. The pricing is a little higher than if you were to use a self-hosted system, but there are so many complicated functionalities it seems well worth paying the extra, as it would save time and reduce complexity.
The robot is capable of connecting to the Internet using Wi-Fi. I was able to slightly increase the effective Wi-Fi range by using a Mikrotik router and altering the hardware retries setting and frame life settings. The intention was to quickly recover from transmission errors and avoid congestion. This discarded video packets that could not be delivered in real time, and kept the network clear for when transmission would be successful. I also used iptables and mangle to alter the DSCP of the live video stream packets with the same intention.
To enable a long range connection, I used Twilio Programmable Wireless to connect to local 4G/LTE cellular networks. I substantially lowered the data rate to around 250 Kbps to make transmission more reliable and reduce costs, and was able to get a virtually flawless live feed.
The live video and audio stream uses FFMPEG for compression and streaming, and has a plethora of settings to tune. I took the time to tune bitrate, buffering, keyframe interval. I also ensured the web camera was able to natively encode video with UVC at the selected resolution to reduce the load on the Raspberry Pi’s CPU. Video latency was often under a second, which is impressive especially considering the round trip involved.
The first 4G/LTE long range mission was successful, and the
webcam was good enough to be used at night. Different members of the community
took turns to drive the robot. It didn’t always drive straight, so we had to
drive forward and turn to the left at regular intervals. The robot drove for around
30 minutes, and then got stuck when it fell down a sidewalk. I had to quickly
drive to retrieve it =)
The second mission was intended to drive from my location to
a friend working at a local business. However half way through the mission, a
suspicious member of the public grabbed the robot, threw it in a trash can, and
called the police. I waited for the police and calmly explained that the robot
was an educational project in telepresence, and also told the person reporting
the robot that there were no hard feelings, despite interfering and damaging my
As part of the community site, it is common to leave your robot open to be controlled. While unattended, a sneaky individual drove my robot into a void of the house and managed to get it covered in spider webs and other filth, as you can see below. Thanks.
I found that cats were very curious about the robot invading their territory, as you can see below:
I was very pleased with how the project worked, and had the opportunity to use Python, Node, and fine-tune wireless networking and live video streaming, and of course remotely control the robot as I had wanted to do for a long time.