Real hardware evidence
Typed objective in. PX4 drone flight out.
Over the last four months I built a natural-language control stack for a real drone. The milestone is simple: one typed mission objective became an outdoor PX4 flight with arm, takeoff, lateral movement, hold, return-to-launch, landing, disarm, and final state verification.
The important part is not how impressive the flight was but what controlled it.
I typed the objective into Codex (or Claude), which called structured MCP
tools exposed by droneserver. From there,
droneserver talked to the aircraft through MAVSDK/MAVLink
over a SiK telemetry radio, PX4 executed the flight actions, and the same
interface checked aircraft state during preflight, flight, and
post-mission return to the launch position.
The video keeps the evidence visible, with tool calls and telemetry on the left and the physical drone on the right. The product direction is a higher-level mission interface that translates operator intent into observable flight mechanics, with the remote controller becoming a safety fallback rather than the primary interface.
Typed objective: take off, move south, hold, return, land.
Codex selects and sequences MCP tools exposed by droneserver.
PX4 executes arm, takeoff, movement, hold, and RTL.
The system confirms landed, disarmed, on ground, and healthy.
Next generation control surface above the aircraft
The drone has not changed; the new thing is the way the user controls it. Instead of beginning with remote controller stick movement or low-level flight commands, the operator can state the mission objective in plain language while the software turns that objective into structured flight actions, constantly checks the drone status, and exposes what it is doing to the user. My custom software is made available to an AI model (testing was done with OpenAI's GPT 5.4) via a Model Context Protocol (MCP) server.
The product idea is not "replace pilots with AI." This is a next-generation way to fly drones, so the operator can spend less time and energy flying and more time deciding what outcome matters. Also keep in mind, flying a drone is hard but describing what you want the drone to accomplish is something anyone can do.
mission objective
tool selection
structured actions
state and telemetry
PX4 interface
real flight
This is software that solves real problems.
Ease of use expands adoption
Skilled drone pilots will always matter, especially when conditions get messy. But many useful drone tasks do not start with a desire to manually manage sticks, modes, and waypoints. They often start with a person who needs to inspect a structure, search a hard-to-reach area, record content, help fight a wildfire, engage in warfare, film nature, and so much more. My prediction is that not every one of these use cases will continue to require a skilled pilot if my software can advance far enough in capability. This software could help increase drone hardware sales, by expanding who the end user can be.
Fills gaps where drone pilots are scarce
A natural-language control layer could also give experienced drone operators an easier tool for routine flight mechanics. That matters most in places where trained pilots are scarce, overloaded, or hard to position quickly. Think about wildland firefighting, search and rescue, infrastructure inspection, and other field operations where aerial evidence is valuable but it's hard to bring in human pilots.
Easier to scale and improve existing pilot efficiency
If you are flying a drone via the remote controller, that means your hands are occupied on the sticks and you can physically only fly one drone at a time. If instead you use my software, you can fly a second drone by opening a second tab. Imagine if every drone operator could double their efficiency by controlling 2 drones at once instead of 1? That seems like software people would pay for.
What the demo supports
- Real PX4 hardware can be reached through an LLM, MCP, MAVSDK, MAVLink, and a SiK telemetry radio.
- The aircraft can execute software-led flight maneuvers that stack together into basic missions.
- The AI-native control path works on real hardware, not just simulation environments.
What I am not claiming
- This is not unattended autonomy yet.
- This is not obstacle avoidance or cluttered-environment navigation yet.
- This is not a mature hardware perception loop yet.
- This is not multi-drone control, BVLOS readiness, or a production deployment model yet.
The next step is external feedback
The milestone is strong enough to take outside the lab. I am using it to start conversations with people building drone hardware, autonomy stacks, simulation environments, field robotics systems, and dual-use mission software.
The questions I care about now are product questions: where does this interface reduce real operator burden, which hardware stack should it be validated on next, and what evidence would make this useful to someone besides me? I think this software has real value, but where do I take it next?
For the full engineering buildout, see Phase Three: Real Hardware and Phase Four: Real Flight.
If you want to discuss the work or compare notes on where this should go next, reach out on LinkedIn. Thank you so much for reading. This is 4 months of hard work I am proud of! -Jake