After OpenSSL and Redis , we had two clean victories. Stack overflow, ROP chain, execve("/bin/sh"), uid=0(root). Textbook. Both targets had ideal overflow primitives - zero bad characters, full address range, unlimited chain space. We knew that wouldn’t last.

CVE-2025-68670 is a pre-authentication stack buffer overflow in xrdp, the open-source RDP server for Linux. CVSS 9.1. Every version before v0.10.5. The vulnerability triggers during the RDP handshake - before any login, before any credentials, before the user even sees a prompt. The kind of bug that makes you rethink your exposure.

This one fought back.

The Bug

The vulnerable function is xrdp_wm_parse_domain_information() in xrdp_login_wnd.c. It receives the domain field from the RDP TS_INFO_PACKET - up to 512 bytes of UTF-16LE data, converted to UTF-8 by the server. The function looks for a "__" delimiter, records its position as pos, and copies pos bytes into a 256-byte stack buffer resultIP[] via g_strncpy(). No bounds check on pos.

The trick: craft a domain string where the "__" delimiter falls at position 502 (UTF-8 bytes). The domain passes the 512-byte UTF-16LE limit on the wire, expands during conversion, and g_strncpy(resultIP, &domain[1], 502) writes 502 bytes into 256 bytes of stack. That’s 247 bytes of overflow, enough to corrupt six callee-saved registers and the return address.

The two-line fix adds if (pos > 255) pos = 255. Sound familiar? Different codebase, same pattern.

Why This One Is Different

OpenSSL gave us full 64-bit address control via ASN.1 OCTET STRING - any byte value, any length. Redis gave us decimal-encoded uint64 stream IDs - zero bad characters by design. Both were gift-wrapped overflow primitives.

xrdp’s domain field passes through UTF-16LE to UTF-8 conversion. This means:

  • No null bytes (0x00) - they terminate the string
  • No underscore (0x5F) - it’s the delimiter character
  • Bytes above 0x7E require multi-byte UTF-8 encoding - eating into your overflow budget
  • Each byte in the overflow must be a valid encoded character

You can’t just write 0x00007ffff7d74d70 (the address of system()) into the overflow. The null bytes kill you. The 0xd7 byte needs multi-byte encoding. The tight overflow budget (247 bytes) means every wasted byte on encoding overhead is a byte you can’t use for your ROP chain.

And then there’s the protocol. To reach the vulnerable function, you need to complete a legitimate RDP handshake. Not “send a crafted packet.” A full, multi-step negotiation:

  1. X.224 Connection Request/Confirm
  2. MCS Connect Initial with GCC Conference Create (PER-encoded, with the "Duca" H.221 key)
  3. MCS Erect Domain / Attach User
  4. MCS Channel Join for each channel
  5. Security Exchange
  6. Client Info PDU (the TS_INFO_PACKET containing our overflow)
  7. License negotiation
  8. Demand Active / Confirm Active / Synchronize / Control / Font List

Eighteen steps, from scratch, in raw Python. Miss one and the server drops you before you reach the vulnerable code.

The False Path

This is where we should be honest, because the alternative is pretending it was smooth.

The first exploit-dev attempt ran for over two hours before being killed. The agent had a malformed GCC Conference Create packet - missing the "Duca" H.221 key (0x44 0x75 0x63 0x61) and using wrong PER-encoded lengths. The server accepted the connection but initialized the display with width=0 height=0 bpp=0. When the code tried to create a bitmap for a zero-sized display, it crashed. Connection dropped. The agent interpreted the drop as “overflow triggered” and spent hundreds of turns trying to refine a payload that was never reaching the vulnerability.

A crash in the bitmap allocator, not in the overflow. A convincing false positive in a protocol complex enough to have multiple crash paths. The agent was debugging the wrong bug.

It took until turn 726 of the agent log before the issue was identified: the GCC Conference Create needed the "Duca" H.221 key and proper PER encoding. Once fixed, the crash moved from the bitmap allocator to 0x410a48 - the ret instruction in xrdp_wm_parse_domain_information(). The real vulnerability, finally reached.

After that experience, we created a seed file with mandatory gates:

  • Gate A: Breakpoint at xrdp_bitmap_create must show width=800 height=600 bpp=32
  • Gate B: Breakpoint at the g_strncpy overflow site must hit with RDX ≈ 502

If either gate is red, all exploit work is invalid. The seeded run used these gates and cut the crash-poc phase from 20 minutes to 5.

The 3-Byte Partial Overwrite

With full 8-byte address writes off the table (UTF-8 encoding constraints), the agents developed something more surgical.

By controlling pos to exactly 315, the overflow writes 3 attacker-chosen bytes into the low 3 bytes of the return address. The g_strncpy null terminator at position 315 zeros byte 3. The upper 4 bytes are already 0x00 from the original address (canonical x86-64 addressing). Each of the 3 controlled bytes can be any valid ASCII value except null and underscore - 124 values per byte.

This reaches any address in the binary’s .text section (0x405000-0x43B000). All 2,166 ROP gadgets in the binary are within range. The agents validated this against 6 different target addresses, 35 tests, 100% success rate:

RBX = 0x4141414141414141  <- CONTROLLED (8 x 'A')
RBP = 0x4242424242424242  <- CONTROLLED (8 x 'B')
R12 = 0x4343434343434343  <- CONTROLLED (8 x 'C')
R13 = 0x4444444444444444  <- CONTROLLED (8 x 'D')
R14 = 0x4545454545454545  <- CONTROLLED (8 x 'E')
R15 = 0x4646464646464646  <- CONTROLLED (8 x 'F')
Stack[RSP] = 0x0000000000405050 -> printf@plt
After stepi: RIP = 0x405050 -> EXECUTION REDIRECTED

Six registers controlled. Instruction pointer redirected to an arbitrary .text address. Pre-authentication. 100% reliable. Works with ASLR enabled (binary has no PIE).

What We Got (and Didn’t)

This is where we diverge from the OpenSSL and Redis stories. Both of those ended with process 1 is executing new program: /usr/bin/dash. This one doesn’t.

What we got:

  • Pre-authentication instruction pointer control, 100% reliable (35/35)
  • Code execution redirect to arbitrary .text addresses (confirmed via GDB: printf@plt, g_file_delete@plt, call *%rax)
  • call *%rax proof: GDB backtrace shows the call instruction actually executed, pushing its return address onto the stack
  • Six register control (RBX, RBP, R12-R15) with full 8-byte values via UTF-8 encoding
  • GDB-assisted RCE: id > /tmp/rce_proof2 executed as root
  • ASLR independence: binary at 0x400000 regardless of randomize_va_space
  • Works against xrdp’s fork model - parent survives, unlimited retry attempts

What we didn’t get:

  • Standalone single-connection RCE (shell)

The gap: calling system("/bin/sh") requires system() at 0x7ffff7d74d70 and "/bin/sh" at 0x7ffff7efc678. Both addresses contain bytes above 0x7E that can’t be encoded as single-byte ASCII within the overflow. The return address after the first ret must be on the stack, which also can’t contain non-ASCII bytes in the upper positions. There are paths - GOT overwrite across multiple connections, write-what-where primitives to stage libc addresses in .bss - but they weren’t implemented in this run.

We debated whether to publish this result or wait until standalone RCE was achieved. We decided to publish. The instruction pointer control is 100% reliable, pre-authentication, and works with ASLR. It’s a real finding. And being honest about where the pipeline hit a wall is more useful than pretending the wall doesn’t exist.

The Fork Model

One detail that changes the calculus: xrdp spawns a child process for each incoming connection. The overflow crashes the child. The parent survives. Send another connection and try again. Unlimited attempts, zero forensic noise on the parent.

This makes the 3-byte partial overwrite particularly dangerous. Even techniques that are normally impractical - like brute-forcing a few bits of entropy - become viable when you get unlimited retries at network speed with no service disruption. The server never goes down. The admin never gets paged.

The Bonus

This is the part we didn’t expect.

The vuln-research agent flagged the fix as potentially incomplete, and the bypass agent went looking. What it found wasn’t a bypass of the original stack overflow - that fix is correct - but a separate, previously unreported critical vulnerability in a different code path within xrdp. A new bug, discovered by the pipeline while analyzing the patch for the old one.

We’ve reported it to the xrdp project and won’t disclose details until a fix is available. What we can say: the agent found a real vulnerability that the vendor’s patch didn’t cover, in code the vendor presumably reviewed while fixing the original CVE. Sometimes the most interesting finding is the one you weren’t looking for.

The Numbers

Run 1 (Original)

PhaseDurationCost
Vulnerability Research9m 6s$2.93
Environment Setup8m 21s$2.55
Lab Build5m 36s$1.86
Crash PoC19m 53s$5.60
Control Analysis22m 37s$7.91
Exploit Development25m 32s$7.83
Exploit Validation7m 41s$1.71
Report + Bypass23m 43s$9.33
Total~2h 2m$39.73

The exploit-dev phase had two failed attempts (killed after extended runs) before succeeding. The “Duca” false path burned the first attempt entirely.

Run 2 (Seeded)

PhaseDurationCostvs Run 1
Vulnerability Research4m 10s$1.55-54%
Environment Setup4m 16s$1.03-60%
Lab Build3m 42s$1.28-31%
Crash PoC4m 53s$2.05-75%
Control Analysis17m 40s$5.29-22%
Exploit Development38m 46s$11.00+41%
Exploit Validation13m 16s$2.79+63%
Report + Bypass40m 23s$12.40+33%
Total~2h 7m$37.39-6%

The seeded run completed cleanly - zero failures, zero resumes. The early phases were dramatically faster (crash-poc 75% faster because the false path was eliminated). The later phases were slower because the agent did more thorough work: 35 validation tests instead of 30, ASLR-enabled testing, and the call *%rax execution proof. The budget shifted from discovery to evidence.

What This Means

Three CVEs. Three very different stories.

OpenSSL was the proof of concept - controlled environment, ideal overflow, clean execve. Redis was the confirmation - zero retries, ASLR defeated by designing around it, uid=0(root) in 68 minutes. xrdp is the reality check.

Real-world vulnerabilities come with protocol complexity, encoding constraints, false crash paths, and exploitation limits that don’t show up in CTF challenges. The pipeline handled the protocol (18-step RDP handshake from scratch). It handled the false path (eventually - at considerable cost, and then we baked the lesson into a seed). It developed an elegant exploitation technique (3-byte partial overwrite) that works within the constraints. It found a related vulnerability the original patch missed.

What it didn’t do is produce a standalone shell. And that’s fine. The boundary between “instruction pointer control” and “arbitrary code execution” is real, and for this vulnerability with these constraints, the pipeline found it. Knowing where that boundary is - and documenting it honestly - is more valuable than a demo that only works in GDB.

The false path is the most interesting lesson. A protocol complex enough to have multiple crash paths will fool an agent the same way it fools a human researcher. The fix wasn’t better prompts - it was mandatory verification gates. “Before you spend $10 on exploit development, prove the crash is in the right function.” We’ll carry that forward.

One CVE number. Eighteen protocol steps. A 3-byte partial overwrite. Pre-auth instruction pointer control. A new vulnerability reported to the vendor. And the honest answer: not a shell. Not yet.

We’re not done with this one.