gets() protection on 8086+

[[e-mail]]

Assuming an application is riddled with gets() or
other problematic code that allows the injection
of machine code by a hacker, is it possible to take
advantage of segmentation to fix this problem?
And perhaps this solution can be carried forward
to the S/380 segmentation.

First assume you only allow these executables onto
your PDOS/86 system:

https://wiki.osdev.org/MZ

If both the minimum and maximum allocation fields are cleared, MS-DOS will attempt to load the executable as high as possible in memory.


This allows you to have all your executables in high
memory, and all your data and stack in low memory.
The 8086+ supports setting an upper and lower
segment number for ds and ss to be set to. Any
attempt to load ds above that value (which would
reach into where the executables are stored) is
greeted with an exception.

You can assume that all data in the protected area
has been completely trashed by hackers taking
advantage of buffer overruns. It all contains deadly
machine code that can exploit bugs in the OS itself
if it is ever executed. The goal then is to ensure this
code is never executed.

So the above protection mechanism also prevents
code from being executed in the data area.

That just leaves function pointers. Segment register
fs: can (and must) point to the protected area, and
when a function is called, and function pointers are
pushed onto the stack, the first thing the called
function does is put those function pointer values
into a crude stack pointed to by fs:. The beginning
of fs: includes a current fs stack pointer and a
maximum stack pointer. Compiler-generated code
will take care of checking those values.

The application does need to be correctly coded
such that function pointers are subject to prototype
checking so that you can't accidentally pass a
random data pointer as a function pointer.

Simple function pointers that are stored in global
variables or whatever are collected into the fs:
area at program initialization.

I can't see any way code in the data area can be
executed, no matter how trashed the data area is
by hackers.

Is that useful?

BFN. Paul.

[Scott Lurndal]

Post by ***@gmail.com
Assuming an application is riddled with gets() or
other problematic code that allows the injection
of machine code by a hacker, is it possible to take
advantage of segmentation to fix this problem?
And perhaps this solution can be carried forward
to the S/380 segmentation.v

It's simple. Descriptor based addressing, to wit, B5500.

https://en.wikipedia.org/wiki/Burroughs_large_systems_descriptors

https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/

[Rod Pemberton]

On Fri, 26 Mar 2021 05:39:47 -0700 (PDT)

Post by ***@gmail.com
Assuming an application is riddled with gets() or
other problematic code that allows the injection
of machine code by a hacker,

a) agree that gets() is problematic
b) not sure I agree that gets() /can/ "inject machine code" ... as it's
known for buffer overflows - it /may/, depending on what is
overwritten, i.e., if data, no, but if code, yes.
c) just redirect gets() etc to a safe function: fgets(), getline() ...

(Ironically, I was once asked if I had created the
"Morris worm", which was an early buffer overflow exploit.
I always find it bizarre when people think low things of me.)

Post by ***@gmail.com
is it possible to take
advantage of segmentation to fix this problem?

a) overkill, complicated, plenty of work, etc?
b) non-portable, i.e., most processors don't have segmentation

Just redirect gets, or wrapper it.

Post by ***@gmail.com
This allows you to have all your executables in high
memory, and all your data and stack in low memory.

And, your OS in low memory, if there is enough room, and
if you want it to be "protected", i.e., place OS outside
application addressable space above 1MB etc.

Post by ***@gmail.com
The 8086+ supports setting an upper and lower
segment number for ds and ss to be set to. Any
attempt to load ds above that value (which would
reach into where the executables are stored) is
greeted with an exception.

Now, you need to code for and handle exceptions for x86
and exceptions aren't portable.

Post by ***@gmail.com
You can assume that all data in the protected area
has been completely trashed by hackers taking
advantage of buffer overruns. It all contains deadly
machine code that can exploit bugs in the OS itself
if it is ever executed. The goal then is to ensure this
code is never executed.

You write code that allows buffer overruns? ... ;-)

Post by ***@gmail.com
So the above protection mechanism also prevents
code from being executed in the data area.

Can't this be done with CS and DS bits for PM? I.e.,
different address ranges for CS and DS segments,
no-execute bit for more recent processors, etc.

Unfortunately, concepts such as memory which is
non-executable is part of Harvard Architecture
design (separate code and data space), while most
processors are of von Neumann architecture design
(shared code and data space). As such, this is
something that is usually implemented in hardware
by design, and not in software, although segmentation
can be used to implement it, e.g., 16-bit x86 PM
and 32-bit x86 PM. I'm not sure if segmentation is
useful for 64-bit x86, as 64-bit x86 eliminates
code and data segments:

"The x86-64 architecture does not use segmentation
in long mode (64-bit mode)." from Wikipedia page
on x86 segmentation:

https://en.wikipedia.org/wiki/X86_memory_segmentation

Post by ***@gmail.com
That just leaves function pointers.

See the "call gate" for x86 PM.