A notable software misconfiguration temporarily allowed users of the Spaghetti Detective to 3D print on other people’s equipment.
The Spaghetti Detective is a cloud-based AI system that uses visual input to determine if a 3D print has failed. The typical failure scenario occurs when a print looses adhesion from the print surface and a giant pile of “spaghetti” filament extrusion results. The Spaghetti Detective would identify this situation and quickly pause the failed print job and send alerts. This also explains the unusual name for the service.
The Spaghetti Detective is implemented as an Octoprint plugin, a popular open source tool allowing intelligent remote control of otherwise “dumb” desktop 3D printers. Octoprint handles the pausing actions when triggered by
But in order to work, participating 3D printers must be controlled via The Spaghetti Detective’s cloud system, where all the AI processing occurs. The cloud system thus accepts requests from all participating systems when they are active.
During a routine software update, techs at The Spaghetti Detective accidentally misconfigured the load balancer that routes incoming requests to their servers. This was unnoticed by all until something very unusual happened.
A Reddit user posted the image at top, saying that it was unexpectedly found one morning on his supposed-to-have-been-idle 3D printer and asked the community what was going on. The mysterious 3D print said:
TSD is not secure
I randomly connected
Sorry had to inform
U
“TSD” means “The Spaghetti Detective”. Evidently a random TSD user was able to connect to the poster’s device and literally initiate — and complete — a 3D print. This is perhaps the most unusual method of informing someone of a security issue as I’ve ever seen.
Predictably, the Reddit community immediately suspected that a super-hacker had someone broken into The Spaghetti Detective and presumably had access to all 3D printers on their network.
The truth, as usual, turned out to be far simpler and less evil: The Spaghetti Detective had made a configuration mistake.
Kenneth Jiang of The Spaghetti Detective quickly realized the error (a misplaced IP address) and rectified the situation. In a long and excellently transparent blog post, Jiang wrote:
“I screwed up. It was the first security breach The Spaghetti Detective has had in 2 years of her existence. But it was an embarrassing one that I can’t forgive myself for.
I made a stupid mistake last night when I re-configured TSD cloud to make it more efficient and run faster. My mistake created a security vulnerability for about 8 hours. The users who happened to be linking a printer at that time were able to see each other’s printer through auto-discovery, and were able to link to them too! We were notified of a case in which a user started a print on someone else’s printer.
73 users got impacted as a result. It’s not a huge number. There are bugs that impact a lot more users. But the consequence is very severe. Nobody wants his/her own printers being linked to and controlled by another account.
I created The Spaghetti Detective to let all 3D printing hobbyists have a way to safely monitor their printers from everywhere. And this is one of the worst mistakes I can make. My sincere apologies to our community for this horrible mistake.”
All 73 users have been contacted by TSD and these participants must freshly reconnect their 3D printers to the network. TSD has also initiated a security audit to identify any similar holes in their network.
This is indeed perhaps the most awful error a cloud service could make, but fortunately this type of service doesn’t seem to expose data that could have been lost. However, having random people run jobs on your 3D printer is a serious matter, particularly if they had ill intent. It might be possible to damage a 3D printer in several ways if a purposefully-designed GCODE file were sent.
The other good news is that the folks at TSD handled the incident about as well as could be hoped. Their transparency and detailed explanations are worthy of review by other organizations to see how it should be done. I encourage you to read their detailed post.
Finally, there’s nothing like an incident to provoke change. In this case it’s highly likely TSD will implement additional procedures that could reduce the probability of similar problems to near-zero.
Via The Spaghetti Detective and Reddit
