write() diagnostics

What was the errno value (or the strerror(errno) string)?

If it was ENOSPC, then yes, the file system was either
out of inodes or had insufficient free space to satisfy
the write(2).

Note that most file systems reserve up to 10% of the free
space for root (e.g. for log files, and so that root can
still log in if the disk is otherwise full). This can
be altered with the tune2fs(2) command if necessary.

Note also that for some file system types (e.g. ufs)
user-based quotas can be enabled. Quotas are always
per file system (as some filesystem types don't support
quotas at all).

You can type:

$ man -k quota

and get all the information you need, if quotas are enabled
and the appropriate tools were installed.

The file system driver in the kernel manages the disk(s)
that contain the file system. No further information
other than ENOSPC will be conveyed to the caller.

Production applications control their disk usage and
production environments are closed environments, no
end-user need apply.

SSD controllers are designed to level the wear across
all cells, and will reallocate spares as necessary, no
software intervention is necessary. As will NVME.

Interactive utilities need just notify the user that
the file system (print the path with the error) was
full and that the user should contact adminstrators
for assistance.

muta...@gmail.com

2021-03-21 21:55:10 UTC

What was the errno value (or the strerror(errno) string)?

Unfortunately I didn't record the errno, only strerror(errno),
which is "No space left on device".

Post by Scott Lurndal
If it was ENOSPC, then yes, the file system was either
out of inodes or had insufficient free space to satisfy
the write(2).

Thanks for the inode pointer. Didn't think of that.

Also thanks for the other info.

Post by Scott Lurndal
The file system driver in the kernel manages the disk(s)
that contain the file system. No further information
other than ENOSPC will be conveyed to the caller.

Ok, thankyou. Do you consider that to be appropriate?
Shouldn't there be secondary analysis logged in
/var/log/something?

Post by Scott Lurndal
Production applications control their disk usage and
production environments are closed environments, no
end-user need apply.

I don't trust that the exact same situation (e.g. running
out of inodes) won't occur in production.

Post by Scott Lurndal
Interactive utilities need just notify the user that
the file system (print the path with the error) was
full and that the user should contact adminstrators
for assistance.

Ok, if you think that is appropriate, that's all I can
actually do anyway it seems. Unless I write my own
OS or something. :-)

Anyway, despite the fact that I ran my last testcase
after 5pm on Friday I think, and when I connected to
my session before 9am Monday, still doing a nano edit
of a Java crash analysis, and did nothing at all except
exit nano and run the exact same test (which had been
consistently failing), the problem disappeared.

So it seems that is the end of that venture.

BFN. Paul.

muta...@gmail.com

2021-03-22 10:13:48 UTC

Post by ***@gmail.com
So it seems that is the end of that venture.

But I have a new problem.

system("sort ...") is returning -1. The documentation
says that that's what happens if fork() or whatever
fails. But I need more information than that to solve
the problem. Is some memory structure corrupt
such that fork() is not seeing the word "sort" or is
there not enough virtual memory (how much do I
need?), or do I not have permission to the "sort"
executable or what?

How should that one work?

Thanks. Paul.

Scott Lurndal

2021-03-22 14:37:08 UTC

Post by ***@gmail.com
So it seems that is the end of that venture.

But I have a new problem.
system("sort ...") is returning -1. The documentation

'returns -1 and sets errno'.

muta...@gmail.com

2021-03-23 02:56:04 UTC

Post by ***@gmail.com
So it seems that is the end of that venture.

But I have a new problem.
system("sort ...") is returning -1. The documentation

'returns -1 and sets errno'.

I re-checked the man page for system() and there is no
mention that errno is set.

However, I did confirm that errno is being set, and is
equal to 12 which is "Cannot allocate memory"
(according to strerror()).

So - next I need to know how much memory it was
trying to allocate, and how much my program is
already using, and how much more memory is
actually available, as per my quota or which disk
is being used for swap, or main memory. Something
like that.

This information is not available to my application,
but I think it should be logged to /var/log so that
an admin can take a look at what memory allocation
is being attempted, and if there is some sort of
corruption where it is trying to allocate 500 TB, we
know that the solution to this problem is not to add
more swap space.

Or if my application was actually in a loop doing malloc()
until it ran out of memory, so Linux has tried so hard to be
accommodating and I am using 100 GB already, but this
meant there was not even 50k left for system() to work,
then we know we need to address the application problem,
it is not a swap size issue at all.

We may as well have an errno that says "a problem
occurred", because the current limited information is
not much better.

BFN. Paul.

muta...@gmail.com

2021-03-23 05:41:29 UTC

Post by ***@gmail.com
Or if my application was actually in a loop doing malloc()
until it ran out of memory, so Linux has tried so hard to be
accommodating and I am using 100 GB already, but this
meant there was not even 50k left for system() to work,
then we know we need to address the application problem,
it is not a swap size issue at all.

Ok, from further research it appears that this is down
to a fundamental flaw in Unix.

Instead of doing a spawn() like normal people do, so
that calling system() only takes 100k or whatever to
execute a small external program, it instead does a
fork(), duplicating a massive address space, only to
then blow it all away with exec().

There's not enough swap space available to do that
I believe. (I'm not really sure how to check that).

Other fundamental flaws in Unix are that you can't
do a "allocate memory" system call, and you can't
get (and set) a sensible parameter to programs,
you need to split it up into individual arguments and
place that on the stack. No thanks.

BFN. Paul.

Scott Lurndal

2021-03-23 14:19:55 UTC

Ok, from further research it appears that this is down
to a fundamental flaw in Unix.

Oh?

Post by ***@gmail.com
Instead of doing a spawn() like normal people do, so
that calling system() only takes 100k or whatever to
execute a small external program, it instead does a
fork(), duplicating a massive address space, only to
then blow it all away with exec().

Linux does, of course, support posix_spawn(2). system()
is defined (and has been for almost 50 years) to call fork()
and exec() internally by the relevent standards.

However, modern unix operating systems use CoW when duplicating
the VA space in fork(), so if nothing is touched between the fork and the
exec, the only cost is to duplicate the page tables for the
new process.

Swap space is only allocated when needed, not at fork().

Scott Lurndal

2021-03-23 14:16:44 UTC

Post by ***@gmail.com
So it seems that is the end of that venture.

But I have a new problem.
system("sort ...") is returning -1. The documentation

'returns -1 and sets errno'.

I re-checked the man page for system() and there is no
mention that errno is set.

https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html

See the section entitled ERRORS.

Post by ***@gmail.com
However, I did confirm that errno is being set, and is
equal to 12 which is "Cannot allocate memory"
(according to strerror()).

As the system(3) library function calls fork(2) and exec(2)
internally, any error that can be returned by fork(2) can be
returned by system(3).

Generally ENOMEM can apply to primary memory, backing store (swap space)
or memory for kernel data structures.

It's likely that the kernel will have logged a diagnostic into the
kernel logs (e.g. dmesg, /var/log/syslog, et alia) when the memory
issue was encountered.

It's also possible that your application encountered a
resource limit (stack, VA space, RSS, etc), 'man setrlimit'.

Post by ***@gmail.com
So - next I need to know how much memory it was
trying to allocate, and how much my program is
already using, and how much more memory is
actually available, as per my quota or which disk
is being used for swap, or main memory. Something
like that.

cat /proc/<pid>/maps
$ cat /proc/$$/maps
00400000-0055f000 r-xp 00000000 08:02 1198460 /usr/bin/ksh
0075e000-0076b000 r--p 0015e000 08:02 1198460 /usr/bin/ksh
0076b000-00771000 rw-p 0016b000 08:02 1198460 /usr/bin/ksh
00771000-00778000 rw-p 00000000 00:00 0
00970000-00972000 rw-p 00170000 08:02 1198460 /usr/bin/ksh
3a4e600000-3a4e620000 r-xp 00000000 08:02 1180249 /usr/lib64/ld-2.18.so
3a4e81f000-3a4e820000 r--p 0001f000 08:02 1180249 /usr/lib64/ld-2.18.so
3a4e820000-3a4e821000 rw-p 00020000 08:02 1180249 /usr/lib64/ld-2.18.so
3a4e821000-3a4e822000 rw-p 00000000 00:00 0
3a4ea00000-3a4ebb4000 r-xp 00000000 08:02 1180267 /usr/lib64/libc-2.18.so
3a4ebb4000-3a4edb3000 ---p 001b4000 08:02 1180267 /usr/lib64/libc-2.18.so
3a4edb3000-3a4edb7000 r--p 001b3000 08:02 1180267 /usr/lib64/libc-2.18.so
3a4edb7000-3a4edb9000 rw-p 001b7000 08:02 1180267 /usr/lib64/libc-2.18.so
3a4edb9000-3a4edbe000 rw-p 00000000 00:00 0
3a4ee00000-3a4ee03000 r-xp 00000000 08:02 1181278 /usr/lib64/libdl-2.18.so
3a4ee03000-3a4f002000 ---p 00003000 08:02 1181278 /usr/lib64/libdl-2.18.so
3a4f002000-3a4f003000 r--p 00002000 08:02 1181278 /usr/lib64/libdl-2.18.so
3a4f003000-3a4f004000 rw-p 00003000 08:02 1181278 /usr/lib64/libdl-2.18.so
3a4f600000-3a4f705000 r-xp 00000000 08:02 1180316 /usr/lib64/libm-2.18.so
3a4f705000-3a4f905000 ---p 00105000 08:02 1180316 /usr/lib64/libm-2.18.so
3a4f905000-3a4f906000 r--p 00105000 08:02 1180316 /usr/lib64/libm-2.18.so
3a4f906000-3a4f907000 rw-p 00106000 08:02 1180316 /usr/lib64/libm-2.18.so
3a63c00000-3a63c02000 r-xp 00000000 08:02 1202927 /usr/lib64/libutil-2.18.so
3a63c02000-3a63e01000 ---p 00002000 08:02 1202927 /usr/lib64/libutil-2.18.so
3a63e01000-3a63e02000 r--p 00001000 08:02 1202927 /usr/lib64/libutil-2.18.so
3a63e02000-3a63e03000 rw-p 00002000 08:02 1202927 /usr/lib64/libutil-2.18.so
7f02b67d9000-7f02b6869000 rw-p 00000000 00:00 0
7f02b6869000-7f02bcd92000 r--p 00000000 08:02 1445663 /usr/lib/locale/locale-archive
7f02bcd92000-7f02bcdde000 rw-p 00000000 00:00 0
7f02bcdf5000-7f02bcdfe000 rw-p 00000000 00:00 0
7ffe1f52b000-7ffe1f54c000 rw-p 00000000 00:00 0 [stack]
7ffe1f594000-7ffe1f596000 r--p 00000000 00:00 0 [vvar]
7ffe1f596000-7ffe1f598000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]

$ cat /proc/$$/statm
29342 670 513 351 0 280 0

/proc/[pid]/statm
Provides information about memory usage, measured in pages. The columns are:

size (1) total program size
(same as VmSize in /proc/[pid]/status)
resident (2) resident set size
(same as VmRSS in /proc/[pid]/status)
share (3) shared pages (i.e., backed by a file)
text (4) text (code)
lib (5) library (unused in Linux 2.6)
data (6) data + stack
dt (7) dirty pages (unused in Linux 2.6)

$ man proc | col -b > /tmp/proc.txt && vim /tmp/proc.txt

More information than you can shake a stick at.

muta...@gmail.com

2021-03-24 01:55:38 UTC

Post by ***@gmail.com
system("sort ...") is returning -1. The documentation

'returns -1 and sets errno'.

I re-checked the man page for system() and there is no
mention that errno is set.

https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html
See the section entitled ERRORS.

Ok, thanks. Maybe I need to contact Linus and ask
him to update his man page?

Post by Scott Lurndal
It's likely that the kernel will have logged a diagnostic into the
kernel logs (e.g. dmesg, /var/log/syslog, et alia) when the memory
issue was encountered.

Ok, good. That's what I'm after. I don't have access
to that, but so long as someone does, that's all I'm
after, so that we can diagnose a misbehaving
application without an application change, and
without the problem needing to be immediately
repeatable.

That's what makes sense, right?

I do have "dmesg" but didn't immediately notice how it
reported *past* out-of-memory situations which is what
I need.

Post by Scott Lurndal
It's also possible that your application encountered a
resource limit (stack, VA space, RSS, etc), 'man setrlimit'.

Thanks.

cat /proc/<pid>/maps

All these things you gave me appear to be used to diagnose
a running process. That is not what I am after. I am after
diagnostics of what happened to an application at the
point that it *terminated* due to insufficient memory to
run system("sort").

Post by Scott Lurndal
Linux does, of course, support posix_spawn(2). system()
is defined (and has been for almost 50 years) to call fork()
and exec() internally by the relevent standards.

The standard that matters to me - C90 - doesn't care
if system() is implemented via posix_spawn(), so next
time I am targeting Unix with PDPCLIB I will switch to
using posix_spawn() to provide a better, at least in this
situation, C library than the one currently being used.

Post by Scott Lurndal
However, modern unix operating systems use CoW when duplicating
the VA space in fork(), so if nothing is touched between the fork and the
exec, the only cost is to duplicate the page tables for the
new process.

It's not the cost that matters to me, it's the numbers.

Post by Scott Lurndal
Swap space is only allocated when needed, not at fork().

I'm not sure what terminology you are using, but:

https://stackoverflow.com/questions/1367373/python-subprocess-popen-oserror-errno-12-cannot-allocate-memory

you have to (in order of preference) either (1) add more swap, and/or (2) relax the overcommit policy (so that the OS lets you fork a large process like python even if very little free memory is available on the "promise" that an exec of a much smaller process will immediately follow the fork), and/or (3) reduce your script's memory footprint so fork doesn't have to as for as much memory on your behalf (chase down memory leaks and bloat, etc.), and/or (4) spawn a helper process, use posix_spawn/vfork, etc.

I believe number 2 is required to get what you are describing.

I certainly don't believe that there wasn't enough memory to
do a posix_spawn() of "sort" (a 99k executable - just checked).

My problem was that the address space had already loaded
GB worth of data (I believe), and fork() wants to make sure
it can duplicate that *if required*.

BFN. Paul.

Scott Lurndal

2021-03-24 14:34:35 UTC

system("sort ...") is returning -1. The documentation=20

=20

'returns -1 and sets errno'.=20

=20
I re-checked the man page for system() and there is no=20
mention that errno is set.

https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html=20
=20
See the section entitled ERRORS.

Ok, thanks. Maybe I need to contact Linus and ask
him to update his man page?

Good luck with that. Linus manages the kernel. Not the distributions.

Rod Pemberton

2021-03-24 04:09:59 UTC

On Tue, 23 Mar 2021 14:16:44 GMT

Post by ***@gmail.com
I re-checked the man page for system() and there is no
mention that errno is set.

https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html
See the section entitled ERRORS.

Um, how do you get from "may set" errno to "is set" or "does set"?

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

muta...@gmail.com

2021-03-24 06:01:25 UTC

Post by Scott Lurndal
https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html
See the section entitled ERRORS.

See the section NOT entitled ERRORS.

Post by Rod Pemberton
Um, how do you get from "may set" errno to "is set" or "does set"?

This bit is more emphatic - "shall" - that's as strong as they get in
language standard, they never seem to add "and if you don't, we're
going to rape your wife, kick your dog and steal your car":

If a child process cannot be created, or if the termination status for the command language interpreter cannot be obtained, system() shall return -1 and set errno to indicate the error

I didn't understand this bit though:

If command is a null pointer, system() shall return non-zero to indicate that a command processor is available, or zero if none is available. [CX] [Option Start] The system() function shall always return non-zero when command is NULL

There's a difference between a "null pointer" and NULL,
and they behave differently?!

Glad I never upgraded from C90!

BFN. Paul.

Scott Lurndal

2021-03-24 14:39:43 UTC

Post by ***@gmail.com
If command is a null pointer, system() shall return non-zero to indicate that a command processor is available, or zero if none is available. [CX] [Option Start] The system() function shall always return non-zero when command is NULL

This is a mechanism for an application to detect whether a shell
(e.g. command processor) is available on the implementation.

Some non-unix operating systems have implemented a subset of the posix
API, and this was a nod to those operating systems (some of which
process commands in the operating system rather than a shell
user process). Most of those systems are obsolete now.

Post by ***@gmail.com
There's a difference between a "null pointer" and NULL,
and they behave differently?!

No, there is a difference in behavior of the system function
when the [CX] option is supported by the implementation.

Scott Lurndal

2021-03-24 14:36:45 UTC

Post by Rod Pemberton
On Tue, 23 Mar 2021 14:16:44 GMT

Post by ***@gmail.com
I re-checked the man page for system() and there is no
mention that errno is set.

https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html
See the section entitled ERRORS.

Um, how do you get from "may set" errno to "is set" or "does set"?

The standards very carefully differentiate between may and shall.

If there is no error from fork, errno may not change.

Note that an implementation, per POSIX, is free to return additional
errno values not described by the standard, so long as the standard
errno values are returned for the conditions noted in the standard.

Rod Pemberton

2021-03-24 22:45:40 UTC

On Wed, 24 Mar 2021 14:36:45 GMT

Post by Rod Pemberton
On Tue, 23 Mar 2021 14:16:44 GMT

Post by ***@gmail.com
I re-checked the man page for system() and there is no
mention that errno is set.

https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html
See the section entitled ERRORS.

Um, how do you get from "may set" errno to "is set" or "does set"?

The standards very carefully differentiate between may and shall.
If there is no error from fork, errno may not change.
Note that an implementation, per POSIX, is free to return additional
errno values not described by the standard, so long as the standard
errno values are returned for the conditions noted in the standard.

According to your explanation, fork() "may set" errno or not, but
system() "must set" whatever fork() has set. AISI, you're changing the
meaning of "may set" for system() to "must set", and pushing down the
"may set" for system() onto fork().

My understanding of "may set" here says that an implementation of
system() has a choice between a) setting errno based on whatever fork()
set for errno, if anything or b) not setting errno by using fork()'s
errnos. I.e., it's implementation-defined as to whether fork()'s errno
is used - or isn't used - to set system()'s errno, for most instances of
errno.

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

Scott Lurndal

2021-03-25 14:06:07 UTC

Post by Rod Pemberton
On Wed, 24 Mar 2021 14:36:45 GMT

Post by Rod Pemberton
On Tue, 23 Mar 2021 14:16:44 GMT

Post by ***@gmail.com
I re-checked the man page for system() and there is no
mention that errno is set.

https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html
See the section entitled ERRORS.

Um, how do you get from "may set" errno to "is set" or "does set"?

The standards very carefully differentiate between may and shall.
If there is no error from fork, errno may not change.
Note that an implementation, per POSIX, is free to return additional
errno values not described by the standard, so long as the standard
errno values are returned for the conditions noted in the standard.

According to your explanation, fork() "may set" errno or not, but
system() "must set" whatever fork() has set. AISI, you're changing the
meaning of "may set" for system() to "must set", and pushing down the
"may set" for system() onto fork().
My understanding of "may set" here says that an implementation of
system() has a choice between a) setting errno based on whatever fork()
set for errno, if anything or b) not setting errno by using fork()'s
errnos. I.e., it's implementation-defined as to whether fork()'s errno
is used - or isn't used - to set system()'s errno, for most instances of
errno.

There is only one variable called errno (per thread in threaded applications);
system just passes through any errno returned from fork() or exec().

Rod Pemberton

2021-03-22 22:51:32 UTC

On Mon, 22 Mar 2021 03:13:48 -0700 (PDT)

Post by ***@gmail.com
But I have a new problem.
system("sort ...") is returning -1. The documentation
says that that's what happens if fork() or whatever
fails. But I need more information than that to solve
the problem. Is some memory structure corrupt
such that fork() is not seeing the word "sort" or is
there not enough virtual memory (how much do I
need?), or do I not have permission to the "sort"
executable or what?
How should that one work?

If I were debugging this, I'd replace the sort command line with a
single command without parameters that printed to the screen, e.g., "ls"
or "dir" etc, just to determine if system() or spawn() etc is working
correctly. Next, add some parameters. If that doesn't work, it's
likely the parameters/arguments need to be quoted e.g., for
MS-DOS/Windows, and/or need escapes for spaces and semicolons
in Linux/*nix.

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

Scott Lurndal

2021-03-22 14:36:24 UTC

Post by Scott Lurndal
The file system driver in the kernel manages the disk(s)
that contain the file system. No further information
other than ENOSPC will be conveyed to the caller.

Ok, thankyou. Do you consider that to be appropriate?
Shouldn't there be secondary analysis logged in
/var/log/something?

What possible additional information could be logged? The disk
is out of space.

Post by Scott Lurndal
Production applications control their disk usage and
production environments are closed environments, no
end-user need apply.

I don't trust that the exact same situation (e.g. running
out of inodes) won't occur in production.

Fortunately, those who do run production for a living do
trust that they won't run out of inodes, as they've planned
their production environment to be sufficient for their
needs, and most of them use LVM so they can add resources
in advance of future need.

Post by Scott Lurndal
Interactive utilities need just notify the user that
the file system (print the path with the error) was
full and that the user should contact adminstrators
for assistance.

Ok, if you think that is appropriate, that's all I can
actually do anyway it seems. Unless I write my own
OS or something. :-)

There is nothing else the OS can do. It certainly can't
start deleting files willy-nilly to make space.

muta...@gmail.com

2021-03-22 17:37:15 UTC

Post by ***@gmail.com
Ok, thankyou. Do you consider that to be appropriate?
Shouldn't there be secondary analysis logged in
/var/log/something?

What possible additional information could be logged? The disk
is out of space.

How many bytes the application was attempting
to write. In my case I don't trust this OS one iota.
As far as I could tell from the seek() and the
write(), all the application was doing was trying
to update its own file. It wasn't trying to extend it.

Indeed, with all these writes happening, the timestamp
of the file wasn't even changing.

The application is Sqlite in fact.

So yeah, I want to see some proof that the application
is actually at fault. The OS is the one making the claim
that there is no more disk space. It could easily write
to /var/log to justify that claim, including whether it
is inodes that is the issue (and how many of them
were available, and how many were needed), or whether
it was quota that was kicking in. This doesn't make
sense to you?

I am confident that some people who run production
workloads, e.g. a bakery, don't even know what an
inode is.

Post by Scott Lurndal
Interactive utilities need just notify the user that
the file system (print the path with the error) was
full and that the user should contact adminstrators
for assistance.

Ok, if you think that is appropriate, that's all I can
actually do anyway it seems. Unless I write my own
OS or something. :-)

There is nothing else the OS can do. It certainly can't
start deleting files willy-nilly to make space.

That is not correct. An OS could exactly bring forward
scheduled cleanup to make space. I'm not saying it
should, I'm just saying options exist, and to me it seems
a little odd that you think the way things currently work
cannot be improved on.

Post by ***@gmail.com
system("sort ...") is returning -1. The documentation

'returns -1 and sets errno'.

Thanks for that! Maybe I missed that in the man page.
I'll see if that gets me further.

BFN. Paul.

Scott Lurndal

2021-03-22 20:15:30 UTC

Post by ***@gmail.com
Ok, thankyou. Do you consider that to be appropriate?
Shouldn't there be secondary analysis logged in
/var/log/something?

What possible additional information could be logged? The disk
is out of space.

How many bytes the application was attempting
to write.

The application provides that to the system call. So it
already has it. Note the return value of the write system
call indicates how many bytes were written by the call.

If part of the requested write can be completed, that
much might be written and the number of written bytes
will be returned. Generally a successful write returns
the value of the 'count' parameter, so one can check

char buffer[512];
char *bp = buffer;
size_t amount_to_write = sizeof(buffer);

while (amount_to_write > 0) {
ssize_t amount_transferred = write(fd, bp, amount_to_write);
if (amount_transfered != amount_to_write) {
/* A short write or an error */
if (amount_transferred == -1) {
/* An error; 'errno' will contain the error code. No data was transferred */
break;
}
}
amount_to_write -= amount_transferred;
bp += amount_transferred;
}

A subsequent attempt to write will return -1 (errno=ENOSPC).

Post by ***@gmail.com
In my case I don't trust this OS one iota.
As far as I could tell from the seek() and the
write(), all the application was doing was trying
to update its own file. It wasn't trying to extend it.

If there are stretches of data in a file that has
never been written (and thus the contents are implicitly
zero), the operating system will never have allocated
sectors to that portion of the file. It will be allocated
the first time that portion of the file is written.

If you opened a new file (O_CREAT) and wrote 512 bytes at
offset zero, and 512 bytes at offset (1<<32)-512, your
EOF would indicate that the file is 4GB, but there will
only be two sectors allocated to the file (or more likely
8x2 to match the 4k page size of the kernel buffers).

Any subsequent write (before closing or after close/reopen)
to any other portion of the file will result in an allocation
and can thus fail with ENOSPC.

I am confident that some people who run production
workloads, e.g. a bakery, don't even know what an
inode is.

Any bakery that runs production will not be using a
PC to control production. They'll be running SAP
or other enterprise software.

A mom-n-pop shop will either use off the shelf PC software
or purchase services from a service provider.

That's not production.

Post by Scott Lurndal
Interactive utilities need just notify the user that
the file system (print the path with the error) was
full and that the user should contact adminstrators
for assistance.

Ok, if you think that is appropriate, that's all I can
actually do anyway it seems. Unless I write my own
OS or something. :-)

There is nothing else the OS can do. It certainly can't
start deleting files willy-nilly to make space.

That is not correct. An OS could exactly bring forward
scheduled cleanup to make space.

That's not a function of the operating system. It's a function
of software that runs on the operating system and how it
is configured.

And no production shop _ever_ would stand for "bringing forward
scheduled cleanup". It's scheduled for a reason.

muta...@gmail.com

2021-03-22 21:02:42 UTC

Post by ***@gmail.com
Ok, thankyou. Do you consider that to be appropriate?
Shouldn't there be secondary analysis logged in
/var/log/something?

What possible additional information could be logged? The disk
is out of space.

How many bytes the application was attempting
to write.

The application provides that to the system call. So it
already has it.

That is true. But the application might not be writing
out that information. It might just be saying "write
error", or it might not even be checking the return code
from the write at all, so is silently misbehaving in
production. And it might be 3rd-party software too.

The question you asked was "what possible additional
information could be logged?". And the answer to that
is "everything". The OS should be recording the fact that
an application (possibly badly-written) has managed to
run out of disk space. At this time, for this amount, on
which filename, and what offset, and whether it was a
"sparse file". And whether it was an inode issue, a quota
issue or whatever.

/* A short write or an error */

There's such a thing as a "short write"? Surely if you
run out of disk space you need an error, ie errno
set to ENOSPC?

The data file I am dealing with was transferred from
the PC to Unix via WinSCP, so I'm assuming there is
no sparse file involved. However, I'm not 100% sure
that it hasn't been extended since then, with sparse
involved.

A mom-n-pop shop will either use off the shelf PC software
or purchase services from a service provider.
That's not production.

I consider it to be production.

Note that I'm in the business of producing software,
e.g. GCCMVS. I consider that to be production. I have
customers who are interested in that working. I don't
consider the fact that they don't pay me to be relevant.
Maybe tomorrow one of them will cut me a check for
$1. The fact that I would then be being paid, but a low
amount, is also logically irrelevant.

And if I'm running Linux instead of Windows, as plenty
of my peers do, then I would be in a position of not
knowing whether I have run out of inodes or not.

There is nothing else the OS can do. It certainly can't
start deleting files willy-nilly to make space.

That is not correct. An OS could exactly bring forward
scheduled cleanup to make space.

That's not a function of the operating system.

Says who?

It's a function
of software that runs on the operating system and how it
is configured.
And no production shop _ever_ would stand for "bringing forward
scheduled cleanup". It's scheduled for a reason.

That is not correct. I have seen people purge the
JES2 spool early because spool has filled up and
the OS won't even start up properly.

Regardless, unless you want to quibble about definitions,
I can totally get my production enterprise and my OS to
delete stuff from c:\scratch on a disk full situation, so that
the application I am running is not aware there was a problem.

Whether I actually choose to do that or not is irrelevant.
The fact is there are options that can theoretically be
configured, even if you don't personally approve of them,
and even if you're convinced that no-one in the world
(not even a bakery in Africa) would ever wish to do something
like that.

BTW, my c:\scratch has a minimum of 8 GB in it. Enough
for any C compile of mine to go through, even if my hard
disk has been filled with porn. And yes, I've seen a production
site (operations) watching porn videos - real videos though,
not occupying hard disk space. And no, it wasn't even in
Africa.

BFN. Paul.

Rod Pemberton

2021-03-22 22:53:48 UTC

On Mon, 22 Mar 2021 20:15:30 GMT

Post by Scott Lurndal
If there are stretches of data in a file that has
never been written (and thus the contents are implicitly
zero), the operating system will never have allocated
sectors to that portion of the file. It will be allocated
the first time that portion of the file is written.

Which OS(es) are you guys talking about that does an allocate-on-write?

Post by Scott Lurndal
Any bakery that runs production will not be using a
PC to control production. They'll be running SAP
or other enterprise software.

I don't know about a bakery, but a small electronics manufacturer I
worked for used Windows 3.1 for employees computers, used MS-DOS and
BASIC in production to operate factory equipment and program EEPROMs,
and ran Unix System V on an x86 PC running an enterprise application
coded in COBOL. The enterprise application handled human resources,
purchasing, accounting, and costing/allocation of parts for
manufacturing. I was told it had a suite of business modules that
could be installed, and was the same program that had been used by a
local Fortune 500 company for decades. Of course, this was the mid
1990's, and we had to fly in a retired COBOL programmer every time
there was some issue.

This was very different from the production at a billion-dollar stock
brokerage I worked for which ran on Stratus Continuum's (in PL/1) for
stock trading, and IBM AS400's for customer accounts, and used Windows
NT PC's for employees and terminals. That was early 2000's but it seems
they were still using those systems for many years after I was gone up
to about 2008 to 2010.

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

Rod Pemberton

2021-03-22 22:51:26 UTC

On Mon, 22 Mar 2021 10:37:15 -0700 (PDT)

Post by ***@gmail.com
How many bytes the application was attempting
to write. In my case I don't trust this OS one iota.
As far as I could tell from the seek() and the
write(), all the application was doing was trying
to update its own file. It wasn't trying to extend it.

Update may not necessarily be an update-in-place without allocating
more space. Sometimes update means write-out-a-new-temporary-file,
delete-the-old-file, and rename. The latter could result in an
out-of-space issue, if there isn't enough space for two copies of the
file, e.g., editing a large file while almost out of disk space.
Certain *nix editors seem to store a change file and an updated file,
without deleting the original file until after exiting.

I know you said you had an out-of-space error. But, what you described
really sounded a lot like a text EOF character/sequence was reached in
the middle of binary file opened as text. I.e., the binary file was
opened as text, and a text EOF was detected, long prior to reaching the
actual binary EOF for the file.

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

muta...@gmail.com

2021-03-24 00:38:58 UTC

Post by ***@gmail.com
Anyway, despite the fact that I ran my last testcase
after 5pm on Friday I think, and when I connected to
my session before 9am Monday, still doing a nano edit
of a Java crash analysis, and did nothing at all except
exit nano and run the exact same test (which had been
consistently failing), the problem disappeared.
So it seems that is the end of that venture.

Guess what? Yesterday (Tuesday) the problem returned,
so I was able to return to debugging SQLite.

Here is the actual original error I received (from sqlite-jdbc):

SQL state [null]; error code [13]; [SQLITE_FULL] Insertion failed because database is full (database or disk is full); nested exception is org.sqlite.SQLiteException: [SQLITE_FULL] Insertion failed because database is full (database or disk is full)

There was also a lot of Java nonsense in response to that.

The database in question is 2.5 GB in size. But there is
37 GB of free space on that file system according to
"df -m". So even if SQLite wants to make a complete
copy of the database it shouldn't be a problem. Again,
unless there's some sort of quota kicking in. And again,
the filestamp of the database isn't actually changing,
it's back in January, so I was surprised that it was able
to write to the database without leaving a footprint.

Anyway, after building my own libsqlitejdbc.so with
my changes to sqlite3.c (amalgamation) I was able
to find out what was really happening.

It needs to write a temporary file! That file is written
to either SQLITE_TMPDIR, TMPDIR or /var/tmp. Since
I didn't have either of those environment variables
defined, it was going to /var/tmp, and the amount of
space there was reasonably small (due to /tmp being
95% full, there was only something like 95 MB
available) and SQLite was trying to write a quite large
file (100 MB I think), and now it makes sense.

If I had been alerted to the fact that a 100 MB temporary
file was being written to /var/tmp because I hadn't
defined either SQLITE_TMPDIR or TMPDIR, I would have
been able to quickly resolve the problem.

Only the application can possibly know about those two
environment variables being used, but the OS (Linux)
should have also been able to report that a write()
error had occurred because I was trying to write 4096
bytes at offset 100 MB to /var/tmp/whatever and there
was only 99 MB space available if we ignore my
application file itself.

The problem should be addressed from both directions, no?

Or how do you think it should work?

Post by Scott Lurndal
What was the errno value (or the strerror(errno) string)?

Unfortunately I didn't record the errno, only strerror(errno),
which is "No space left on device".

I confirmed that it was 28, ENOSPC.

BFN. Paul.

Scott Lurndal

2021-03-24 14:31:52 UTC

Guess what? Yesterday (Tuesday) the problem returned,
so I was able to return to debugging SQLite.
If I had been alerted to the fact that a 100 MB temporary
file was being written to /var/tmp because I hadn't
defined either SQLITE_TMPDIR or TMPDIR, I would have
been able to quickly resolve the problem.
Only the application can possibly know about those two
environment variables being used, but the OS (Linux)
should have also been able to report that a write()
error had occurred because I was trying to write 4096
bytes at offset 100 MB to /var/tmp/whatever and there
was only 99 MB space available if we ignore my
application file itself.

How will the operating system report such a thing? There's no
reporting channel for this.

The application is 100% responsible for reporting and
correcting (if possible) errors. The application should
have reported the path name of the file for which the
ENOSPC indication was reported.

muta...@gmail.com

2021-03-24 16:57:07 UTC

Post by ***@gmail.com
Only the application can possibly know about those two
environment variables being used, but the OS (Linux)
should have also been able to report that a write()
error had occurred because I was trying to write 4096
bytes at offset 100 MB to /var/tmp/whatever and there
was only 99 MB space available if we ignore my
application file itself.

How will the operating system report such a thing? There's no
reporting channel for this.

That seems like an odd question to me - make one!

I don't know why /var/log/syslog can't be used to
report this, if it's already apparently reporting out-of-memory
issues.

Other options would be to run it under control of a
program like "catchsegv" to instead "catch-syscall-failures".

Another option would be to have an environment variable
VERBOSE_SYSCALL_FAILURES.

Another option would be for the C runtime library to set
a global variable auto_errors. write() is in fact in a library
before it invokes a syscall(), so it has the opportunity to
do a fprintf(stderr) when it gets a return of -1. Just before
it does that fprintf it should do another syscall() called
GET_LAST_ERROR_AS_VERBOSE_STRING which returns
a pointer to a 10,000-byte character array with a very long
explanation of what the application just tried to do, and
the preferred contact, possibly phone number, of the
person who can add more disk space to the file system
in question that just got filled up. Or do an inode-reorganization
if that is what is required.

We had an interesting problem in MVS shops. The manuals
all said things like "application programmer response - try
xyz and if it still fails, contact your systems programmer".
What I wanted to see what a "systems programmer response -
try abc and if it still fails, contact IBM support".

BFN. Paul.

Scott Lurndal

2021-03-24 18:19:23 UTC

How will the operating system report such a thing? There's no
reporting channel for this.

That seems like an odd question to me - make one!

Unix has been in use for half a century. If someone thought it
would have been a useful feature, it would have been added by
now.

The application has everything it needs to provide accurate,
timely and fully informative error messages, including the path
name of the file (which the kernel won't necessarily have at
the point at which the kernel detects the out-of-space
condition). Note that pathnames aren't tied to the file content
in any subtantial way, and multiple pathnames can refer to the
exact same file via either hard links or soft links.

Post by ***@gmail.com
We had an interesting problem in MVS shops. The manuals
all said things like "application programmer response - try
xyz and if it still fails, contact your systems programmer".
What I wanted to see what a "systems programmer response -
try abc and if it still fails, contact IBM support".

IBM wasn't only mainframe manufacturer. I spent 14
years writing operating systems for Burroughs, and we
also had such manuals. We also sold our machines for
seven figures, so we could afford to generate such
documentation, and due to the relative cost of disk
space and rarity of memory space, applications weren't
always able to provide useful diagnostics in many
conditions. For example, if an application running
under Burroughs MCP couldn't allocate disk, it would
print a terse message on the operators console and the
operator could then take what steps were necessary
(e.g. wait a while for space to free, explicitly free up space,
add a new pack to a pack family,
discontinue the application, et alia).

Similarly, if the application attempted to open a
a file that didn't exist, the applcation would be suspended
until the file was present (eg. loaded from a tape)
or the operator specified an alternate file name to use.

There are no operators monitoring the workload 24x7
anymore.

Many of the proprietary unix distributions (aix, hpux,
digital unix, SGI IRIX, SunOS/Solaris) also had significant
user documentation in this area.

The Mainframe systems were comparatively primitive
(timesharing, if exists, was batch-with-patch). One
can hardly look at them as modern state-of-the-art.

Linux is a volunteer effort, so the documentation is
also volunteer driven (based generally off the standards
documents with added descriptions of implementation specific
behavior). It's also incomplete, as you've noted. Feel
free to update the pages and submit them for consideration
to the package sponsor.

muta...@gmail.com

2021-03-24 18:41:28 UTC

How will the operating system report such a thing? There's no
reporting channel for this.

That seems like an odd question to me - make one!

Unix has been in use for half a century. If someone thought it
would have been a useful feature, it would have been added by
now.

Another odd statement. I think it would be useful.
Am I not someone? If so, the latter doesn't follow
from the former.

The application has everything it needs to provide accurate,

1. No, the application doesn't have everything it needs.
Some information is hidden by the OS.

2. Not all applications are written to the standards you
and I apparently expect, and the OS has the ability to
compensate for this, and thus, in my opinion, it should.

timely and fully informative error messages, including the path
name of the file (which the kernel won't necessarily have at
the point at which the kernel detects the out-of-space
condition).

It can store it if it wants. I just checked PDOS/386
and it does NOT store the filename. However, I don't
see any reason why I can't change that, and I can't
see any reason why I can't or shouldn't implement
the functionality I described to compensate for
applications such as SQLite that are issuing
insufficient diagnostics.

Although I can't do a mandatory log to stderr,
because SQLite does some deliberate writes
that it deliberately doesn't care if they fail, before
eventually doing one that it cares about, because
it has real data instead of a single NUL character.

documentation, and due to the relative cost of disk
space and rarity of memory space, applications weren't
always able to provide useful diagnostics in many
conditions.

If they were C programs short of space, you could
#ifdef the diagnostics out, right?

For example, if an application running
under Burroughs MCP couldn't allocate disk, it would
print a terse message on the operators console and the
operator could then take what steps were necessary
(e.g. wait a while for space to free, explicitly free up space,
add a new pack to a pack family,
discontinue the application, et alia).

Isn't this something that should be hidden by either
the C90 library or the OS? In hindsight, obviously.
That's what I'm after. If I go back in a time warp to
1940, I want to write something down to help future
programmers and systems designers. Currently I
don't know what to write down. Other than "C90
seems pretty good, but add a '\e' for the ESC
character".

BFN. Paul.

Scott Lurndal

2021-03-24 19:43:33 UTC

How will the operating system report such a thing? There's no
reporting channel for this.

That seems like an odd question to me - make one!

Unix has been in use for half a century. If someone thought it
would have been a useful feature, it would have been added by
now.

Another odd statement. I think it would be useful.
Am I not someone? If so, the latter doesn't follow
from the former.

The application has everything it needs to provide accurate,

1. No, the application doesn't have everything it needs.
Some information is hidden by the OS.

Be specific. Exactly _what_ information is hidden by the OS?

Post by ***@gmail.com
2. Not all applications are written to the standards you
and I apparently expect, and the OS has the ability to
compensate for this, and thus, in my opinion, it should.

As pointed out, the OS doesn't have all the information
that the application does about its intent.

timely and fully informative error messages, including the path
name of the file (which the kernel won't necessarily have at
the point at which the kernel detects the out-of-space
condition).

It can store it if it wants.

No, it cannot. Many files don't even have a name, and many
have multiple names.

Post by ***@gmail.com
I just checked PDOS/386
and it does NOT store the filename. However, I don't
see any reason why I can't change that, and I can't
see any reason why I can't or shouldn't implement
the functionality I described to compensate for
applications such as SQLite that are issuing
insufficient diagnostics.
Although I can't do a mandatory log to stderr,
because SQLite does some deliberate writes

It's open source. Fix it and submit a patch upstream.

This is all too abstract to have a useful discussion about.

Please suggest exact (not vague it should keep filenames)
data that the kernel should maintain in order to produce
useful error diagnostics for the following conditions:

- Space cannot be allocated for a write or pwrite system call to a regular file
- Memory cannot be allocated for a malloc (recalling, of course, that malloc is
not an operating system function, but rather a C library function)
- The application has exceeded a resource limit (CPU time, RSS, VA, et alia)

All of these can be detected (and potentially repaired) by the application.

While they can be detected by the OS, the OS can do nothing but either
provide the error status to the application (as it does today) or terminate the application
(rather unfriendly, don't you think?). Most users don't have access to
any OS logs (e.g. syslog) for security reasons, so there is no way for the
OS to communicate to an enduser other than via the application the user
is running. Hence errno and signals.

muta...@gmail.com

2021-03-25 10:04:20 UTC

Post by Scott Lurndal
The application has everything it needs to provide accurate,

1. No, the application doesn't have everything it needs.
Some information is hidden by the OS.

Be specific. Exactly _what_ information is hidden by the OS?

Whether the ENOSPC (the only diagnostic available
to the app) was caused by:

1. Insufficient inodes
2. Quota exceeded (and exact details of that)
3. Insufficient free FAT clusters.

Post by ***@gmail.com
2. Not all applications are written to the standards you
and I apparently expect, and the OS has the ability to
compensate for this, and thus, in my opinion, it should.

As pointed out, the OS doesn't have all the information
that the application does about its intent.

It may not have ALL of it, but it has SOME of it,
including all the parameters to write(), and in the
absence of all applications reporting all
parameters to write(), the OS has the ability to
step in to compensate, and I can't see any reason
why it shouldn't.

Post by Scott Lurndal
timely and fully informative error messages, including the path
name of the file (which the kernel won't necessarily have at
the point at which the kernel detects the out-of-space
condition).

It can store it if it wants.

No, it cannot. Many files don't even have a name, and many
have multiple names.

I don't know what you're talking about. Both fopen()
and open() take a filename. You can't ask for a
random file to be opened. Regardless, if you have
some esoteric use case that is doomed to bypass
the OS, so be it. If the OS can report the exact cause,
plus filename, of 99.9999% of fopens, that sounds
pretty bloody good compared to the current situation
of reporting 0% of them.

And who cares about multiple names (softlinks)?
You only need one name to diagnose the problem.
Instead of 0 names, your preferred alternative for
reasons you haven't explained, but sound a lot like
NIH.

It's open source. Fix it and submit a patch upstream.

Interestingly the SQLite people don't actually accept
patches, because they don't trust that the submitted
code is genuine public domain.

It is my intention to at least report the problem to
them, but I'd like to negotiate with you what the
actual problem is, before submitting a problem
report.

I'd like to know what both the application and the
OS should ideally be doing to run applications
with integrity. Once I understand that, I'll fire off
one or two problem reports.

This is all too abstract to have a useful discussion about.

I don't know why you're saying that. I have a specific
line of code in SQLite that is failing in my opinion,
and seemingly your opinion too (not abstract), and
I am ALSO wondering what the general principles are
that should be in play here, especially given that this
is an OS newsgroup!!!

Please suggest exact (not vague it should keep filenames)

I have no idea why you are calling that vague. What on
earth is vague about the filename that is passed on
fopen()/open()/syscall_open()?

data that the kernel should maintain in order to produce
- Space cannot be allocated for a write or pwrite system call to a regular file

Name of executable, name of API call, non-vague filename,
offset of file, how many bytes were attempted to be written
at that offset, whether the file was sparse, how much free
space was actually on the disk at the time at the time of
failure, whether there were sufficient inodes, whether any
quota was in effect, and exact details (noting that the
sysadmin may have since changed the quota settings),
date (ISO format), time (24-hour), UTC offset.

- Memory cannot be allocated for a malloc (recalling, of course, that malloc is
not an operating system function, but rather a C library function)

Only for crappy operating systems.

Regardless - what "quota" restrictions were in place on
this application's use of virtual memory. Confirmation
that it is in fact virtual memory, not real memory or
esoteric memory being requested, what the theoretical
maximum virtual memory space was available to the
application at that time, given the size of the swap
space (which was what?) and how much was being
used by other people (how much was that?), and how
much memory was actually being requested, and how
much memory had already been allocated to your
address space, and how much of that was being
consumed by executables, in case the actual situation
was some sort of rogue posix_spawn gone mad, and
confirmation that the out-of-memory condition was
an actual malloc() rather than a system()/fork() trying
to do a ridiculous duplication of the address space.

- The application has exceeded a resource limit (CPU time, RSS, VA, et alia)

How many CPU seconds it consumed. Start time and
end (cancel) time of the process. Name of executable,
including path. Name of user. Name of cwd. Whether
the job was cancelled by the OS or a sysadmin (and
if the latter, his userid and name), and if it was the OS,
the exact setting that was responsible for that, so the
sysadmin can potentially be asked to increase it, or
the sysadmin can suggest a different job class to run
the job under. How much virtual memory was in use
at the time. What files (full names!) were open. Stack
trace or at least offset in executable.

All of these can be detected (and potentially repaired) by the application.

C90 provides no such option. My primary interest is in
how to implement C90 to the maximum extent possible.

I'm not totally wedded to C90 though, I'm willing to
countenance a C90+ so long as it can be implemented
on all platforms that C90 can be implemented on.
E.g. I see no reason why '\e' can't be added to represent
ESC. If you want to extend the measly errno variable, a
single integer, as if all of the above information can be
represented in an integer (?!), then let's discuss that.
Perhaps a new function so that applications can do:

if (errno != 0)
{
print_os_diagnostics();
}

An implementation is allowed to make that a macro
that does nothing, of course, so it will still work on
a Commodore 64. But if Linus gets his act into gear,
print_os_diagnostics() will give you the name, home
phone number and address of the bastard sysadmin
who put a stupid quota restriction on your userid at
5pm on Friday.

While they can be detected by the OS, the OS can do nothing but either
provide the error status to the application (as it does today)

It provides one friggin integer today.

or terminate the application
(rather unfriendly, don't you think?).

Not really. That sounds like a failsafe to me, and should
probably be configurable.

Most users don't have access to
any OS logs (e.g. syslog) for security reasons,

Why do you always think that everything is stuck in
stone for eternity? This is software. This is an OS
group. Anything you can do with a C compiler is
possible.

In this specific case, how about a Unix command
"show_my_syslog" which is setuid root and retrieves
all information from the syslog that is generated by
a process you own? Or have an environment variable
SHOW_MY_SYSLOG which causes the OS to write
to both /var/log/syslog AND your home directory in
a file called syslog.txt.

so there is no way for the
OS to communicate to an enduser other than via the application the user
is running. Hence errno and signals.

So little imagination! Only Linus can invent things?

BFN. Paul.

Scott Lurndal

2021-03-25 14:19:12 UTC

Post by Scott Lurndal
The application has everything it needs to provide accurate,

1. No, the application doesn't have everything it needs.
Some information is hidden by the OS.

Be specific. Exactly _what_ information is hidden by the OS?

Whether the ENOSPC (the only diagnostic available
1. Insufficient inodes
2. Quota exceeded (and exact details of that)
3. Insufficient free FAT clusters.

What is a FAT cluster? Oh, you're referring to the long
obsolete microsoft filesystem. Well, that filesystem,
being close to a half-century old is very much not state-of-the-art.

What would the application do differently if it knew that the
problem was due to quota exceeded (which is signaled by
errno == EDQUOT)?

What would the application do differently if it new that the
problem was due to insufficient fat clusters?

Post by ***@gmail.com
It may not have ALL of it, but it has SOME of it,
including all the parameters to write(),

The _APPLICATION_ has the parameters to write.

Post by ***@gmail.com
and in the
absence of all applications reporting all
parameters to write(), the OS has the ability to
step in to compensate, and I can't see any reason
why it shouldn't.

Time, space, complexity. Those are the three reasons
that the OS should have no part in diagnosing application
failures.

No, it cannot. Many files don't even have a name, and many
have multiple names.

I don't know what you're talking about. Both fopen()
and open() take a filename.

1) fopen/open are not the only ways to open a file.
2) Not all files represent objects on a "disk".
3) Multiple directory entries can point to the same INODE,
in which case a file has multiple names.
4) All directory entries for a file can be removed yet
so long as it is open by at least one process, the
file will continue to exist in the filesystem without
any name.
5) Sockets (network endpoints) are represented by file descriptors
and have no name. Memory based files are represented by file
descriptors and have no name.
6) Why should the kernel waste an arbitrary amount of space (PATH_MAX)
per each open file for something the application already knows?

Post by ***@gmail.com
And who cares about multiple names (softlinks)?

And if that particular symbolic link (or hard link)
was removed after the application started but before
the application encountered an error?

Post by ***@gmail.com
You only need one name to diagnose the problem.
Instead of 0 names, your preferred alternative for
reasons you haven't explained, but sound a lot like
NIH.

The guiding principle of any operating system is to
keep it simple. Don't add unnecessary cruft. Keeping
file pathnames around (up to RLIMIT_NOFILE) would be
a waste of space, time and add unneeded kernel complexity.

Modern kernels are _far_ more complex than DOS, whether it
be MSDOS, PDOS or IBM 360 DOS.

Scott Lurndal

2021-03-25 15:03:00 UTC

Post by Scott Lurndal
The application has everything it needs to provide accurate,

1. No, the application doesn't have everything it needs.
Some information is hidden by the OS.

Be specific. Exactly _what_ information is hidden by the OS?

Whether the ENOSPC (the only diagnostic available
1. Insufficient inodes
2. Quota exceeded (and exact details of that)
3. Insufficient free FAT clusters.

What is a FAT cluster? Oh, you're referring to the long
obsolete microsoft filesystem. Well, that filesystem,
being close to a half-century old is very much not state-of-the-art.
What would the application do differently if it knew that the
problem was due to quota exceeded (which is signaled by
errno == EDQUOT)?

To elaborate on this; from the write(2) manual page:

ERRORS
EAGAIN The file descriptor fd refers to a file other than a socket and has been marked nonblocking (O_NONBLOCK), and the write would block.

EAGAIN or EWOULDBLOCK
The file descriptor fd refers to a socket and has been marked nonblocking (O_NONBLOCK), and the write would block. POSIX.1-2001 allows either
error to be returned for this case, and does not require these constants to have the same value, so a portable application should check for
both possibilities.

EBADF fd is not a valid file descriptor or is not open for writing.

EDESTADDRREQ
fd refers to a datagram socket for which a peer address has not been set using connect(2).

EDQUOT The user's quota of disk blocks on the file system containing the file referred to by fd has been exhausted.

EFAULT buf is outside your accessible address space.

EFBIG An attempt was made to write a file that exceeds the implementation-defined maximum file size or the process's file size limit, or to write at
a position past the maximum allowed offset.

EINTR The call was interrupted by a signal before any data was written; see signal(7).

EINVAL fd is attached to an object which is unsuitable for writing; or the file was opened with the O_DIRECT flag, and either the address specified in
buf, the value specified in count, or the current file offset is not suitably aligned.

EIO A low-level I/O error occurred while modifying the inode.

ENOSPC The device containing the file referred to by fd has no room for the data.

EPIPE fd is connected to a pipe or socket whose reading end is closed. When this happens the writing process will also receive a SIGPIPE signal.
(Thus, the write return value is seen only if the program catches, blocks or ignores this signal.)

Other errors may occur, depending on the object connected to fd.

muta...@gmail.com

2021-03-28 19:36:53 UTC

Post by ***@gmail.com
Whether the ENOSPC (the only diagnostic available
1. Insufficient inodes
2. Quota exceeded (and exact details of that)
3. Insufficient free FAT clusters.

What is a FAT cluster? Oh, you're referring to the long
obsolete microsoft filesystem. Well, that filesystem,
being close to a half-century old is very much not state-of-the-art.

Public domain code is lagging behind virus-licensed
code, so it is state-of-the-art in its own field.

What would the application do differently if it knew that the
problem was due to quota exceeded (which is signaled by
errno == EDQUOT)?

Not the application, the end user. If the application
passes on this information to the user, they know
they need to ask for a quota restriction to be lifted,
not more disks to be bought. Well obviously in both
cases they pass this info on to the sysadmin. Along
with exactly how much data they were trying to
write at the point the quota was exceeded. Because
if that amount is 1 EiB, it's more likely to be a bug
than a too-low quota.

What would the application do differently if it new that the
problem was due to insufficient fat clusters?

Pass that different information on to the user who
passes that on to the sysadmin. Actually, the
sysadmin shouldn't be trusting a report from an
end user (and user mode application) like that in
the first place, and have an independent reference
in /var/log/syslog that is from a more reliable
source (the OS). So that "I couldn't even write 5
bytes!" is converted into "Actually you couldn't
write 5 EiB, which is not that surprising".

Post by ***@gmail.com
It may not have ALL of it, but it has SOME of it,
including all the parameters to write(),
The _APPLICATION_ has the parameters to write.
and in the
absence of all applications reporting all
parameters to write(), the OS has the ability to
step in to compensate, and I can't see any reason
why it shouldn't.

Time, space, complexity. Those are the three reasons
that the OS should have no part in diagnosing application
failures.

Ok, that's an interesting position. Can you elaborate
on that? What time? CPU time? It is only in case of
error that a syslog entry would be written. Space?
Disk space? In 2021? Complexity? What percentage
increase in the Linux source code would occur if it
started doing something really useful like logging
exact cause of an application system call failure?

I have PDOS/386 as a reference and now that I have
been alerted to a real-world problem I don't really see
any barrier to activating syscall logging in PDOS. Are
you predicting it will double the source code base or
what?

No, it cannot. Many files don't even have a name, and many
have multiple names.

I don't know what you're talking about. Both fopen()
and open() take a filename.

1) fopen/open are not the only ways to open a file.

So? The real world example I have presented does
indeed go via open(). I'm not expecting to solve
world poverty with this change, I'm expecting to
cover a particular class of problem that caused
me grief. In 2021.

2) Not all files represent objects on a "disk".

See above.

3) Multiple directory entries can point to the same INODE,
in which case a file has multiple names.

Either filename will be fine.

4) All directory entries for a file can be removed yet
so long as it is open by at least one process, the
file will continue to exist in the filesystem without
any name.

Not a problem. The name will tell me what file system
it did/still does exist on.

5) Sockets (network endpoints) are represented by file descriptors
and have no name. Memory based files are represented by file
descriptors and have no name.

See the world poverty thing.

6) Why should the kernel waste an arbitrary amount of space (PATH_MAX)
per each open file for something the application already knows?

Because some applications are coy about letting
people know that. Even something as famous as
SQLite just gives a vague "database or disk full".

Post by ***@gmail.com
And who cares about multiple names (softlinks)?

And if that particular symbolic link (or hard link)
was removed after the application started but before
the application encountered an error?

And what if aliens come to earth and put a gun to
my head and say "if you print out the name of that
file, I will kill you"? I'm trying to solve the most
common problem, not get every single last possible
problem solved.

Post by ***@gmail.com
You only need one name to diagnose the problem.
Instead of 0 names, your preferred alternative for
reasons you haven't explained, but sound a lot like
NIH.

The guiding principle of any operating system is to
keep it simple.

That would be PDOS/386. As opposed to Linux. Let
me know when you've made the transition.

Don't add unnecessary cruft. Keeping
file pathnames around (up to RLIMIT_NOFILE) would be
a waste of space, time and add unneeded kernel complexity.

It's not a waste, and it's not complex either. It's
just "NIH". It is the most obvious way to solve
this real-world problem I was presented with
in 2021. The alternative solution of fix every
application in existence is not nearly as viable,
and won't work anyway, because only the OS
knows about things like inodes.

Modern kernels are _far_ more complex than DOS, whether it
be MSDOS, PDOS or IBM 360 DOS.

So much for "kernel complexity", eh?

Tell me what's really getting your goat and we can
move on. You have shares in Red Hat? Don't like
the DD/MM/YY date format?

BFN. Paul.

Rod Pemberton

2021-03-26 06:07:14 UTC

On Thu, 25 Mar 2021 03:04:20 -0700 (PDT)

You can't ask for a random file to be opened.

see tmpfile() and tmpnam()

Interestingly the SQLite people don't actually accept
patches, because they don't trust that the submitted
code is genuine public domain.

Good for two reasons: copyrights, control of code quality.

My primary interest is in
how to implement C90 to the maximum extent possible.

Why?

While a full C library is convenient for coding
full-featured applications for an OS, you only need
the C language proper (C89) and stdio.h to bootstrap
code. Everything else is trivial to re-implement or
unnecessary. A slightly less rudimentary situation
would be the C language proper (C89), stdio.h, stdlib.h
and string.h. (FYI, that last combination is all the C
that I used for coding my OS, plus some assembly, both
inlined in C and standalone.)

I'm not totally wedded to C90 though, I'm willing to
countenance a C90+ so long as it can be implemented
on all platforms that C90 can be implemented on.

Offhand, I'm not sure what is really needed from C99
or C2K etc, other than "long long" and the ability to
printf() long-long format, i.e., "%ll" or "%L" etc.

[...] will give you the name, home
phone number and address of the bastard sysadmin
who put a stupid quota restriction on your userid at
5pm on Friday.

Is this at work or school? You're on a roll. So, don't
rage quit on us just yet. We need someone to talk to
about something other than religion.

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

muta...@gmail.com

2021-03-26 21:49:40 UTC

You can't ask for a random file to be opened.

see tmpfile() and tmpnam()

Both of those wind up calling syscall_open() with an
actual filename that the OS knows about, and could
remember if it wanted to so that it could print out
the details later when it runs out of space.

My primary interest is in
how to implement C90 to the maximum extent possible.

Why?

It's a starting point. To see what is possible. ie can
you print the sysadmin's home address when the
quota is exceeded, using purely C90?

Post by Rod Pemberton
While a full C library is convenient for coding
full-featured applications for an OS, you only need
the C language proper (C89) and stdio.h to bootstrap
code.

I'm not only talking about the OS.

The application can ALSO print the sysadmin's
home address.

If you think that is appropriate. And again, within
the confines of C90, or at least, C90+.

Post by Rod Pemberton
Everything else is trivial to re-implement or
unnecessary. A slightly less rudimentary situation
would be the C language proper (C89), stdio.h, stdlib.h
and string.h. (FYI, that last combination is all the C
that I used for coding my OS, plus some assembly, both
inlined in C and standalone.)

Why did you feel the need to use inlined assembler,
which makes your code non-C90-compliant?

I'm not totally wedded to C90 though, I'm willing to
countenance a C90+ so long as it can be implemented
on all platforms that C90 can be implemented on.

Offhand, I'm not sure what is really needed from C99
or C2K etc, other than "long long" and the ability to
printf() long-long format, i.e., "%ll" or "%L" etc.

I wouldn't even take that. I don't think applications
should be relying on a long long long long long
to be implemented on a Commodore 64, as
programmers get increasingly lazy.

If you have 512-bit registers, just make long 512 bits.

[...] will give you the name, home
phone number and address of the bastard sysadmin
who put a stupid quota restriction on your userid at
5pm on Friday.

Is this at work or school?

Neither. It's a theoretical problem.

Post by Rod Pemberton
You're on a roll. So, don't
rage quit on us just yet. We need someone to talk to
about something other than religion.

I'm not sure what "rage quit" is. I have left this group,
for years at a time, because I was still trying to
understand the mainframe, and the Hercules groups
were more appropriate.

BTW, less than a week ago I got to the point where
I could start Hercules, which automatically started
PDOS/3X0, and then I was able to use micro-emacs,
all from a Windows command prompt. No separate
EBCDIC ANSI terminal needed to be attached.

Thanks largely due to a recompilation with Cygwin gcc
and its inbuilt ANSI emulation.

There are still some caveats though, but I think they
can be overcome.

BFN. Paul.

Rod Pemberton

2021-03-27 00:05:55 UTC

On Fri, 26 Mar 2021 14:49:40 -0700 (PDT)

Post by ***@gmail.com
The application can ALSO print the sysadmin's
home address.
If you think that is appropriate.

A relative is a privacy attorney. So, I have to say
it's inappropriate in this age of personal data privacy,
and would only have been appropriate up to the mid 1990's.

Why did you feel the need to use inlined assembler,
which makes your code non-C90-compliant?

x86 has all sorts of special instructions for entering
and using protected mode (PM). The data structures
needed by some of these instructions e.g., selectors
and descriptors are easy to create and use from C. So,
I only, with one exception, used standalone assembly for
larger blocks of assembly code. Also, you can call the
inlined assembly as C functions, pass C data to them,
apply special compiler directives to the functions, such
as naked (no prolog or epilog). In other words, yes,
there is part of my OS which is "portable" and part which
is x86 specific. I don't see this as a problem, because
the OS programs the x86 hardware which (x86 PM, keyboard,
mouse, PIC, IRQs, ...) is all non-portable too.

Post by ***@gmail.com
I'm not sure what "rage quit" is.

Gaming term for someone who is upset, angry and leaves,
after being killed in the game. You've seen videos of
people smashing computer keyboards or televisions when
gaming, yes? ...

<OT rage quit example from Q3A>

E.g., many years ago, this guy kept calling me a cheat
a Quake3 Arena. So, he creates another character
actually using a cheat (see through walls, automatic aim,
shot avoidance). He harassed me (and his buddies) for
years, chasing me from my preferred server to other
servers just to kill me repeatedly, until I got upset
and left. This also drove down my ranking. After about
five years or so, he calls me out again with his friends
on the server. Some hidden server admins were present
too. He demands that I explain how I'm killing him if
I wasn't cheating. So, I told him. It was sort of
the end of the Quake3 Arena era anyway. I shot at him
based on the sounds of his footsteps. Then, there was
a heated argument that you couldn't hear footsteps in
the game. You could. They had just turned off footstep
sounds. Once he found out I was killing him by sound,
tracking him by sound, shooting at him by sound, not
cheating, and not using sight to kill him, he got upset,
swore and left. Rage quit. In Q3A, the preferred game
weapon was a "laser rifle" called a "rail gun". I was
exceptionally good with using other weapons, like rockets,
which took a long time to hit their target, which was
moving and running around the map. Using the sound of
footsteps, I could predict when and where you'd be or go.

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

James Harris

2021-03-24 10:37:39 UTC

Post by ***@gmail.com
I have a problem in that Linux x64 write() is returning
-1 when writing to a seemingly innocuous spot of a
disk file.
I confirmed that errno/strerrno changed at this point,
into a message saying that the disk was full.
But I need a lot more information than that to
diagnose the problem. "df -k" doesn't show any
disks full. If this happens in production, what
steps need to be taken by the end user?

You've run into what I would call two programming principles (or, more
accurately, into the lack of application thereof).

The first principle is that errors should be notified at low level and
reported at high level.

What I mean by that is that the low level (in this case part of your
run-time system or your OS) should not just detect the problem but also
prepare a notification of it. Then your program will be free either to
report the error or to work round it. If it reports the error then it
will be able to report all the information that the low level added to
the notification.

In practice that means your program would have been able to report
something like "could not write block B to file F because the write
would have exceeded quota for user U". Reporting that amount of detail
could come at no cost to your program. Your program would just send to
stderr a printable version of the error notification.

The second principle is perhaps more controversial and it is that
programs should allocate space before they begin writing to it. The
space required could be enough for just one record or even one byte -
whatever's about to be written, or it could be for an entire file, but
it would make allocation and writing separate operations.

I say that because without pre-allocations it is not possible in a
multitasking environment to guarantee that storage space available
before one begins writing will be available for exclusive use during the
write. Nor is it sensible to assume that storage space is infinite.

(Preallocate-before-write can have much wider benefits than just
exception handling. I came up with the idea (though it may not have been
new) as a way of achieving two key features I wanted: record-based IO
and lockless simultaneous writes. That's a separate topic but I mention
it just to point out that the approach has benefits beyond the obvious.)

Post by ***@gmail.com
Is there some compromise between Plan A above
and a more realistic Plan B?

I see it as above: An OS should notify at low level and allow reporting
at high level. (It can be especially convenient if a language has
exception handling.) And a program should request space before writing
to it.

--
James Harris

muta...@gmail.com

2021-03-24 11:13:08 UTC

On Wednesday, March 24, 2021 at 9:37:41 PM UTC+11, James Harris wrote:

Hi James. Thanks for your thoughts.

Post by ***@gmail.com
But I need a lot more information than that to
diagnose the problem. "df -k" doesn't show any
disks full. If this happens in production, what
steps need to be taken by the end user?

Currently write() just returns -1 and sets errno (an
integer). Specifically where is the program going to
get all that information about the quota for user U?
Is the OS supposed to provide extended information
beyond the errno? If so, do you have something
specific in mind?

Also, do you agree that the OS should be logging this
independently, regardless of whether the application
"does the right thing"?

I don't trust either OSes or applications, so I'd like
both to do their best.

Post by James Harris
The second principle is perhaps more controversial and it is that
programs should allocate space before they begin writing to it. The

Ok, I don't like the sound of this, so let's leave this
aside for now and concentrate on the above. :-)

Post by James Harris
I see it as above: An OS should notify at low level and allow reporting
at high level. (It can be especially convenient if a language has
exception handling.)

Can you tell me more about this exception handling
you envisage, even if it means some extensions to
C90?

Thanks. Paul.

James Harris

2021-03-25 09:45:09 UTC

Post by ***@gmail.com
Hi James. Thanks for your thoughts.

Hi Paul. You're welcome.

As this group is about OS development my reply was about what I believe
an OS or run-time system /ought/ to do but AIUI you want your OS to be
compatible with ones which already exist and have their own mechanisms
for exception reporting.

You might be best sticking with what you are emulating. That said, there
may be a compromise along the lines you already mentioned: via logging.

Post by ***@gmail.com
Also, do you agree that the OS should be logging this
independently, regardless of whether the application
"does the right thing"?

In principle, yes. I would agree that programs which come across an
exceptional condition should send a message about it. However, it would
then be up to a configurable message handler to determine whether to
write that message to the log or to ignore it or to do something else
with it. Unix syslog is configurable along those lines:

https://linux.die.net/man/5/syslog.conf

...

Post by James Harris
I see it as above: An OS should notify at low level and allow reporting
at high level. (It can be especially convenient if a language has
exception handling.)

Can you tell me more about this exception handling
you envisage, even if it means some extensions to
C90?

Yes, as the exception notification bubbles up the OS return stack it
will eventually be passed up to the user's program. If the language the
program was written in has exception handling then it should be able to
handle the issue it as it would any other exception.

By contrast, as C90 doesn't have automatic exception handling a
programmer has to check for exceptions manually, usually along the lines of

retcode = syscall1();
if (retcode != OK)
{
/* Handle the exception */
}

However, if you want a more informative message than you'd get from
perror or strerror then this is not really a language issue. You'd
really need to alter the operation of the underlying OS.

--
James Harris

muta...@gmail.com

2021-03-25 10:20:19 UTC

Post by James Harris
As this group is about OS development my reply was about what I believe
an OS or run-time system /ought/ to do

And that is *exactly* what I am interested in, even if
I *personally* don't have the resources to implement it.
I'd like to at least *understand* how things *should*
work.

Post by James Harris
but AIUI you want your OS to be
compatible with ones which already exist and have their own mechanisms
for exception reporting.
You might be best sticking with what you are emulating.

I'm not sure where you got that idea from. I have two
tangible OSes at the moment, PDOS/386 and PDOS/3X0,
and it is true that both of them handle, to some extent,
executables + APIs from other OSes. But that was just
meant to be a starting point to begin to understand how
an OS works. It took me a long time to realize that a.out
executables could actually be relocated if they were
built properly. I instead had an elaborate system of
adjusting the base.

And my latest proposal is to replace all that previous
nonsense with a brand new OS that is totally portable,
and dependent only on a brand new BIOS, and that
that BIOS should have callback functions including
fopen()/fread() that access raw hard disks instead of
normal files that an OS deals with.

Post by James Harris
Yes, as the exception notification bubbles up the OS return stack it
will eventually be passed up to the user's program. If the language the
program was written in has exception handling then it should be able to
handle the issue it as it would any other exception.
By contrast, as C90 doesn't have automatic exception handling a

Do you consider this to be a serious shortcoming in C90
that should be rectified? If so, is it possible to rectify it
in a portable manner?

Post by James Harris
programmer has to check for exceptions manually, usually along the lines of
retcode = syscall1();
if (retcode != OK)
{
/* Handle the exception */
}

Is there a problem doing it that way? That's the only way
I've ever coded it. I've never moved beyond C90 so I am
not attached to any alternate paradigm. I wanted to
exhaust C90 first.

Post by James Harris
However, if you want a more informative message than you'd get from
perror or strerror then this is not really a language issue. You'd
really need to alter the operation of the underlying OS.

I don't know why you would say that. Why can't this
be done in the language in a standard manner?
Along the lines of char *get_last_error(); which
potentially returns a 10,000-byte string full of
information that can be dumped to stderr. With the
C90+ standard giving the implementation the
option of returning just "\n" to every call.

Are you perhaps drawing a distinction between
"language" and "OS" that I'm not seeing? Given that
all of C's functionality is in functions, it seems to me
that it is actually impossible to differentiate between
the language and the OS. Is there a tangible difference
between fopen/open/DosOpen?

BFN. Paul.

muta...@gmail.com

2021-03-25 10:25:33 UTC

I don't have time to write commentary tonight, but
here's the latest installment of raw data. I've basically
been meditating in Tibet for the last few decades,
and when I returned to civilization, in *2021*, this is
what I see:

Failed to load native library:sqlite-3.32.3-cfaaf6a4-6366-4cb2-b785-868ad9482e9f-libsqlitejdbc.so. osinfo: Linux/x86_64
java.lang.UnsatisfiedLinkError: /tmp/sqlite-3.32.3-cfaaf6a4-6366-4cb2-b785-868ad9482e9f-libsqlitejdbc.so: /tmp/sqlite-3.32.3-cfaaf6a4-6366-4cb2-b785-868ad9482e9f-libsqlitejdbc.so: failed to map segment from shared object: Operation not permitted

https://man7.org/linux/man-pages/man2/mmap.2.html

EPERM The prot argument asks for PROT_EXEC but the mapped area
belongs to a file on a filesystem that was mounted no-exec.

grep EPERM /usr/include/asm-generic/errno-base.h

#define EPERM 1 /* Operation not permitted */

Good grief. 2021.

BFN. Paul.

Melzzzzz

2021-03-25 10:43:03 UTC

Post by ***@gmail.com
I don't have time to write commentary tonight, but
here's the latest installment of raw data. I've basically
been meditating in Tibet for the last few decades,
and when I returned to civilization, in *2021*, this is
Failed to load native library:sqlite-3.32.3-cfaaf6a4-6366-4cb2-b785-868ad9482e9f-libsqlitejdbc.so. osinfo: Linux/x86_64
java.lang.UnsatisfiedLinkError: /tmp/sqlite-3.32.3-cfaaf6a4-6366-4cb2-b785-868ad9482e9f-libsqlitejdbc.so: /tmp/sqlite-3.32.3-cfaaf6a4-6366-4cb2-b785-868ad9482e9f-libsqlitejdbc.so: failed to map segment from shared object: Operation not permitted
https://man7.org/linux/man-pages/man2/mmap.2.html
EPERM The prot argument asks for PROT_EXEC but the mapped area
belongs to a file on a filesystem that was mounted no-exec.
grep EPERM /usr/include/asm-generic/errno-base.h
#define EPERM 1 /* Operation not permitted */
Good grief. 2021.
BFN. Paul.

You probably coped that to tmofs which is usually mounted no-exec.

--
current job title: senior software engineer
skills: x86 aasembler,c++,c,rust,go,nim,haskell...

press any key to continue or any other to quit...

Scott Lurndal

2021-03-25 14:09:45 UTC

What is your complaint? Everything you need to about the
issue is right here. Perhaps you shouldn't have mounted
the filesystem containing the shared object with the
no-exec flag.

$ mount | grep tmp

muta...@gmail.com

2021-03-25 18:08:47 UTC

Post by ***@gmail.com
Good grief. 2021.

What is your complaint? Everything you need to about the

Wow. You have to ask? You don't see anything wrong
with that?!

issue is right here. Perhaps you shouldn't have mounted

Neither I, nor the end user, did any such thing, nor
do we even know what "mount" means. That's not
our job.

the filesystem containing the shared object with the
no-exec flag.
$ mount | grep tmp

Correct. That is the underlying technical issue. They
couldn't have made it more complicated. Well, maybe
they could have. They could have issued a message
saying:

Your application failed when it executed the OS
call located at 0x00ff3299a6 passing 0x23738
in %rax, which is what really got you. But the
parameter at offset 0x52 on the stack was
involved too, just saying.

Yeah, thanks Linus. I was looking forward to
computers in 2021 too.

Ok, now for the write-up.

This is a real-life situation (in 2021). Only the names
of the users have been changed, to protect the innocent.

An end user running an application is expecting to
receive a message of either "all records successfully
processed" or an error along the lines of "line 2000
in your input file contains non-numeric digits in the
phone number associated with Fred Smith". (And they
are totally trained on how to deal with either scenario
above). Instead, they were presented with:

Failed to load native library:sqlite-3.32.3-cfaaf6a4-6366-4cb2-b785-868ad9482e9f-libsqlitejdbc.so. osinfo: Linux/x86_64
java.lang.UnsatisfiedLinkError: /tmp/sqlite-3.32.3-cfaaf6a4-6366-4cb2-b785-868ad9482e9f-libsqlitejdbc.so: /tmp/sqlite-3.32.3-cfaaf6a4-6366-4cb2-b785-868ad9482e9f-libsqlitejdbc.so: failed to map segment from shared object: Operation not permitted

Plus a shitload more Java messages, I think "beans" got
more than one mention, and "utf8" got a mention (the
input files don't contain utf8, but why let that get in the
way of a good error message?).

If I were the end user, my first reaction would be "Holy Cow,
it's Armageddon!" or at least "Holy Cow, Russian hackers
have taken over the computer system!".

Obviously it's beyond their pay grade, but at some level
they know to report the problem to the programmers who
wrote the application rather than the sysadmin. Even if
they get that wrong, the sysadmins can probably spell
"programmer".

So it winds up at the programmer's desk and the process
can begin.

Note that before I disappeared to Tibet, I was working on
IBM mainframes, in the 1980s, and users would be hit
with either an application error message which both they
and the programmer understood, or they would get
something like:

IKJ1234T - virtual memory exceeded

And they could look that "IKJ1234T" up in a manual and it
would say "user response: do xyz, and if problem persists,
see your systems programmer". My largest complaint was
that, as a systems programmer, IBM hadn't also included
"systems programmer response: do abc, and if problem
persists, report it to IBM".

But no worries, I'm sure computer scientists around the
world will have that sorted out by 2021 while I was busy
yodeling in Tibet.

What I wasn't expecting was that with hardware engineers
making truly ridiculous amounts of CPU power, memory and
hard disk space available, we would take a step so far back
in time we may as well be using vacuum tubes.

So as discussed, the problem has made it to the application
programmer. The application is actually in Java, not C, and
if they had received a message "string index of variable
xyz on line 2321 was out of bounds (15)" they would in
fact know what to do.

Instead, they are presented with ancient Swahili.

Failed to load native library:sqlite-3.32.3-cfaaf6a4-6366-4cb2-b785-868ad9482e9f-libsqlitejdbc.so. osinfo: Linux/x86_64
java.lang.UnsatisfiedLinkError: /tmp/sqlite-3.32.3-cfaaf6a4-6366-4cb2-b785-868ad9482e9f-libsqlitejdbc.so: /tmp/sqlite-3.32.3-cfaaf6a4-6366-4cb2-b785-868ad9482e9f-libsqlitejdbc.so: failed to map segment from shared object: Operation not permitted

Assuming you know to look at this bit, which were
the first messages rather than the last or middle,
you can start diagnosing the problem.

Ok, so maybe they know what "native" means. It is
custom-designed for this computer. But what's a
"library"? Never heard of it. Other than a place that
has books. If you have a specific .java file you are
referring to, I can cope with that. But no library, no
books.

I don't know what "cfaa..." is - that's a strange name
to call your so-called "library". Perhaps at some level
we can guess that we should ignore everything written
in hex, and just focus on the Swahili characters.

There is a specific name - sqlite and jdbc. I'm not sure
if Java programmers are actually aware what packages
provide them with database capabilities, but let's say
they do. They recognize that as "ah, the database has
a problem". And let's further suppose that they know
that they are defacto responsible for the database in this
instance, they can't pass the problem on to the DBA.

Ok, so now we have UnsatisfiedLinkError. Maybe the
Java programmers can look this very strange error up
in the Java manual and be informed that we're back
to that "native library" thing. Ok, so we know that a
native library with a very strange name, full of hex, and
in /tmp (not a known application directory) has some
sort of problem. How did this become our problem?
We are not responsible for what crap Linus put in /tmp.
If the OS is apparently putting some strange internal
file into /tmp to do god knows what, that's not on us.

But let's say Linus is in Tibet himself at the moment and
we need to solve the problem ourselves.

If you look hard enough, you will find libsqlitejdbc.so in
the .jar file, so you can maybe guess that there is some
Java code that knows what to do with this .so file. And
indeed, there is, there's a suite of sqlite-jdbc java code.
Still not my problem, except it became my problem when
I started using freeware that doesn't print error messages
like "IKJ1234T". Ok, at that level I can accept responsibility.
I'm now the proud owner of sqlite-jdbc.

failed to map segment from shared object: Operation not permitted

What does "map" mean? What is a "segment" - I know what
a segment register is, but most people don't. "shared object"?
Never heard of it. "Operation not permitted" - why not, is it
against state or federal law, and why didn't you tell me which
one in the first place instead of being coy?

And that's actually the end of the useful diagnostics. You're
supposed to solve the problem with that.

The next logical step would be to look up the word "failed"
in the error message manual, to see what the user response
to "failed to map segment" actually means. IKJ1234T is a
lot more sensible to look up than "failed", but hey, maybe
Linus isn't very good at counting.

Perhaps the underlying problem here is people making their
production processes dependent on what a bunch of
volunteers randomly choose to volunteer instead of going
to an organization like IBM that has people specifically
hired because they know how to count (just as they did
in the mid 1980s and maybe even earlier).

Or if IBM is too expensive, whoever IBM's competitor is.
But again, someone else who knows how to count.

It so happens that with the aid of the internet, I was
eventually able to isolate the root cause as a call
to mmap() (although I don't know who exactly is
doing that, and noting that it is a function that I
was previously unfamiliar with, and noting that it
isn't even Java, and this is a Java application).

And it turns out that at least on our system, you
can't put executables (and "shared libraries" seem
to count here) in /tmp, because it has been deliberately
configured/mounted to prevent such executables from
being run. If Linus thinks that's a good idea, ok, fine, I
don't care. SO LONG AS he prints out a message saying
"hey sucker, you're not allowed to put executables, or
even .so files like you are using, on /tmp, because YOU
SUCK, so go and put it in a different directory".

I don't mind being insulted, so long as you tell me what
your ridiculous rules are. If you're going to impose such
a rule, then also provide some sort of safeguard. Any
application (so this would be sqlite-jdbc) that writes
out executables to a file system and then attempts to
execute them, should have a way of checking that this
file system hasn't been mounted as no-exec and either
find another file system to write to, or ask the user to
provide one by setting an environment variable, in a
very clear message.

Note that this would be very Linux-specific code, not
C90 or Java, but when you do things like create your
own executables (which is basically what is being
done, even if it's just being copied from somewhere
else, and assuming you're not stuffing up permission
bits in that process), you're writing system-specific
code anyway, so it is reasonable to pay the piper. But
it does make it difficult to future-proof your software
if some jackass OS vendor decides to implement a
new new rule that "you can't execute executables
from any directory with "t" in the name". There needs
to be some new standard, similar to C90, that provides
a way of executing executables on all computer systems,
including MVS.

Irrespective of the standard, one way or another, this is
what should have happened.

The end user is presented with a message:

A system error has occurred. Please pass the below
information on to the vendor (programmer) of your
application which is called "fix_phone_numbers".

(reams of indecipherable crap)

The systems administrator is presented via email or
whatever, plus a message in /var/log/syslog, saying:

User fred executed fix_phone_numbers which internally
attempts to put an executable on /tmp, which, as you
know, you personally mounted as no-exec, so it won't
be allowed, but both the user and the programmer are
obviously going to be too stupid to figure that out for
themselves, so please contact fred on mobile phone
number 1234 5678 and tell him that you are going to
speak to the programmer, bob, and ask him what
mechanism, probably an environment variable, allows
the user to specify what directory temporary
EXECUTABLES are written to, because /tmp is
completely inappropriate.

The programmer is presented (as part of that reams of
crap above) with very clear information that a specific
executable file called libjdbc.so was written to /tmp (which
is allowed) and then attempted to be executed (which is
not allowed, because /tmp is mounted (using the "mount"
command or /etc/config) as "no-exec", preventing
execution). The reason /tmp was chosen is because it is
the default if environment variable TEMP_EXEC is not set.
The reason we're writing out an executable in the first
place is because Linux doesn't have the ability to execute
an executable directly from a .jar file which is what we
would ideally want, and hopefully Linus will get off his
fat ass one day and fix that, with Java being the most
popular language in the world and all that, even though
we should have all stuck with C90, for reasons which I
won't get into at the moment. And also Linux doesn't
provide a syscall to find out whether a directory is on
a filesystem that is mounted no-exec, so we can't warn
you about that in advance and not even bother writing
the executable in the first place, although I guess we
could have done our own mmap() to guess in advance
that this was about to go down. Maybe the sqlite-jdbc
people need to get off their fat asses too.

I'm willing to negotiate, both the exact wording, and add
further information you think might be useful in this situation.

Or am I asking for something that is technically
impossible, even in 2021? Do we need to wait
until we have 128-bit long long longs?

What would you do if you had control of an OS and
could create any standard you wanted? We actually
do live in the free world where you can do exactly
that. The folks at ISO aren't going to kick your door
down because you created a non-ISO standard. You
personally have multiple levels of protection against
ISO's goon squad. Specifically, you have men with
guns that can easily defeat ISO's goons. Therefore,
the sky's the limit. All you need to do is negotiate
with Scott, who seems to think that if Linus doesn't
think something is a good idea, then that's the final
word on all matters. Maybe he just doesn't like
upstarts from the southern hemisphere? Linus is yet
another stinking northerner, after all, so he's "cool"?

BFN. Paul.

Rod Pemberton

2021-03-26 06:07:26 UTC

On Thu, 25 Mar 2021 11:08:47 -0700 (PDT)

Post by ***@gmail.com
Good grief. 2021.

What is your complaint? Everything you need to about the

Wow. You have to ask? You don't see anything wrong
with that?!

[rant, extra-long, snipped]

Now, you're ranting about what M$ did for decades: design single points
of failure into their software, i.e., errors which caused the
application to terminate. It's interesting that engineers are taught
to NOT design in single points of failure, but programmers aren't. M$
seemed to resolve much of this with Windows 10.

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

muta...@gmail.com

2021-03-26 22:03:00 UTC

Post by Rod Pemberton
Now, you're ranting about what M$ did for decades: design single points
of failure into their software, i.e., errors which caused the
application to terminate.

That was not my complaint. I don't care about
applications terminating, I care about the user
being given a very clear message about what
just happened and what they need to do.

And I care about someone besides the user
being informed, so that the user can just stare
at the screen contemplating suicide and
someone else will contact them, tell them "we
know what just happened, and apologize for
it, and we've fixed the problem, just try again
and forget this ever happened".

I just checked my almanac again. Definitely 2021.

Post by Rod Pemberton
It's interesting that engineers are taught
to NOT design in single points of failure, but programmers aren't.

That's an interesting comment.

I think in an aeroplane situation you need to be able
to recover from errors, but as a programmer, I'm far
more interested in isolating bugs than being nice to
users.

If I encounter an unexpected problem, I'm more likely
to have things configured so that the program
immediately freezes, or at least coredumps, the user
is locked out from his terminal so that he can no
longer interfere with the system, and in fact, lock down
the entire file system, and freeze the memory too, and
only allow running an embedded-in-memory read-only
diagnostic tool to be run until I've gathered every bit
of information, including the exact time, I need to
diagnose the problem, or at least, put traces/storage
alter traps in place so that next time the program runs,
I'll have the next bit of information I need.

https://www.chiark.greenend.org.uk/~sgtatham/bugs.html

Actually I would consider not having time available
in the first place. I don't want the environment that
my program runs under to be disturbed. I want it to
be infinitely repeatable.

Post by Rod Pemberton
M$ seemed to resolve much of this with Windows 10.

What do you mean? My C90 programs started behaving
differently when I upgraded from Windows 8 to Windows 10?

BFN. Paul.

Rod Pemberton

2021-03-27 00:06:06 UTC

On Fri, 26 Mar 2021 15:03:00 -0700 (PDT)

Post by Rod Pemberton
Now, you're ranting about what M$ did for decades: design single
points of failure into their software, i.e., errors which caused
the application to terminate.

That was not my complaint. I don't care about
applications terminating, I care about the user
being given a very clear message about what
just happened and what they need to do.

If the application doesn't terminate, still
functioning correctly despite the error, does
the user need any message at all? No. (my point)

Post by Rod Pemberton
It's interesting that engineers are taught
to NOT design in single points of failure,
but programmers aren't.

That's an interesting comment.

...

Post by ***@gmail.com
I think in an aeroplane situation you need to be able
to recover from errors, but as a programmer, I'm far
more interested in isolating bugs than being nice to
users.

Isolating bugs after the fact, or coding in style that
avoids them in the first place? Both? That really
reduces C down to a simple subset, sort of K&R C, and
creates all sorts of bizarre syntax documented among
numerous C style guidelines and automotive MISRA C.

Post by ***@gmail.com
If I encounter an unexpected problem, I'm more likely
to have things configured so that the program
immediately freezes, or at least coredumps, the user
is locked out from his terminal so that he can no
longer interfere with the system, and in fact, lock down
the entire file system, and freeze the memory too, and
only allow running an embedded-in-memory read-only
diagnostic tool to be run until I've gathered every bit
of information, including the exact time, I need to
diagnose the problem, or at least, put traces/storage
alter traps in place so that next time the program runs,
I'll have the next bit of information I need.

Halt switch. What if you had to do none of that because
everything worked perfectly? Is that a good scenario,
i.e., no headaches, or a bad scenario, i.e., loss of job
security?

Post by Rod Pemberton
M$ seemed to resolve much of this with Windows 10.

What do you mean? My C90 programs started behaving
differently when I upgraded from Windows 8 to Windows 10?

I mean Win10 almost never crashes, blue screens, fails, etc.
Unlike earlier versions of Windows, it's almost flawless.
One family member's computer installed perfectly from the
get-go. Another, had a nightmare of a problem installing
(early release) and then had a problem with the keyboard,
but after six months, it was all fixed. The newer machine
fully updates without rebooting. The older one has long
reboot time for large updates. I haven't had to deal with
fixing problems with their machines in years now. No need
to run anti-virus software. No need to fix the OS or repair
system files. The only remaining issue is that one machine
must boot into safe mode after a large update to properly
re-detect three USB printers (normal, address, photo).

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

muta...@gmail.com

2021-03-28 02:05:29 UTC

Post by Rod Pemberton
I mean Win10 almost never crashes, blue screens, fails, etc.
Unlike earlier versions of Windows, it's almost flawless.

Wow. Ok, my Win 10 64-bit is running on a stock
standard Dell laptop. I have no peripherals, not
even a USB stick.

I have 85 GB free disk space at the moment, meaning
my virtual memory should be basically infinite. I have
8 GiB of RAM.

I have very little software installed. Certainly not hardware
drivers other than Dell-provided ones that are auto-installed.

I do have Cygwin installed (32-bit), and Visual Studio 2008
32-bit, and Google Chrome and Windows Live Mail. And
Facebook Messenger and VLC.

I do have a ridiculously large number of Google Chrome
windows and tabs open, and a ridiculous number of
emails in my inbox in Windows Live Mail.

An hour or so ago, when I was trying to swap screens,
it would take about 30-60 seconds and sometimes
land on a completely random screen other than the
one I selected.

While writing my long message in comp.lang.c I regularly
copied my message into notepad and saved it as
c:\scratch\temp.txt, as I have done for decades, because
I was expecting to lose everything as Win 10 finally gave up.

I should probably be doing my editing using micro-emacs
in the first place rather than taking my chances that
google groups under Google Chrome is going to survive
under Windows 10, but I like a challenge.

I've once had micro-emacs freeze too, so nothing is
totally safe.

At the same time as this was happening, Windows Live
Mail conked out with an out-of-memory error I haven't
seen before. Like I said, I have infinite virtual memory,
but admittedly this is a 32-bit app, so it's 4 GiB at best,
not infinite.

I chose "continue", was able to copy and paste my email,
quit my message without saving, quit Windows Live Mail,
start it again, and do a fresh reply, pasting in my partial
reply, and it all worked.

I was also able to save the Windows Live Mail crash data,
see below.

I haven't rebooted since this, and the system seems to be
more stable at the moment, but I'm not changing screens
a lot. So if you wish to ask me any questions, I can probably
answer them if you get in before either Google Chrome
finally crashes, or Windows 10 kills it.

And people still ask me why I'm writing PDOS.

BFN. Paul.

See the end of this message for details on invoking
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.Collections.Stack..ctor()
at WindowsLive.Writer.Mshtml.MarkupPointer..ctor(MshtmlMarkupServices markupServices, IMarkupPointerRaw pointer)
at WindowsLive.Writer.Mshtml.MshtmlMarkupServices.CreateMarkupPointer()
at WindowsLive.Writer.Mshtml.MshtmlMarkupServices.CreateMarkupRange()
at WindowsLive.Writer.Mshtml.MarkupRange.Clone()
at WindowsLive.Writer.Mshtml.MarkupHelpers.AdjustMarkupRange(IHTMLTxtRange& stagingTextRange, MarkupRange range, Int32 offset, Int32 length)
at WindowsLive.Writer.SpellChecker.SpellingHighlighter.ProcessWordRange(MshtmlWordRange wordRange)
at WindowsLive.Writer.SpellChecker.SpellingHighlighter.DoWork()
at WindowsLive.Writer.SpellChecker.SpellingHighlighter.CheckSpelling(MshtmlWordRange range)
at WindowsLive.Writer.SpellChecker.SpellingManager.HighlightSpelling(MarkupRange range)
at WindowsLive.Writer.SpellChecker.SpellingManager.DamagedRange(MarkupRange range, Boolean doSpellCheck)
at WindowsLive.Writer.PostEditor.PostHtmlEditing.BlogPostHtmlEditorControl.HandleSpellingDamage(Object source, DamageEvent evt)
at WindowsLive.Writer.HtmlEditor.WordRangeDamager.OnDamageOccured(DamageEvent e)
at WindowsLive.Writer.HtmlEditor.WordRangeDamager.FireDamageOccurred()

************** Loaded Assemblies **************
mscorlib
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.9151 (WinRelRS6.050727-9100)
CodeBase: file:///C:/Windows/Microsoft.NET/Framework/v2.0.50727/mscorlib.dll
----------------------------------------
WindowsLive.Writer.PostEditor
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLive.Writer.PostEditor.dll
----------------------------------------
WindowsLive.Writer.Interop
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLive.Writer.Interop.DLL
----------------------------------------
WindowsLive.Writer.Mshtml
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLive.Writer.Mshtml.DLL
----------------------------------------
WindowsLive.Writer.HtmlEditor
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLive.Writer.HtmlEditor.DLL
----------------------------------------
WindowsLive.Writer.ApplicationFramework
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLive.Writer.ApplicationFramework.DLL
----------------------------------------
WindowsLive.Writer.CoreServices
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLive.Writer.CoreServices.DLL
----------------------------------------
WindowsLive.Writer.Interop.Mshtml
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLive.Writer.Interop.Mshtml.DLL
----------------------------------------
WindowsLive.Writer.Passport
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLive.Writer.Passport.DLL
----------------------------------------
System
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.9149 (WinRelRS6.050727-9100)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System/2.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.9149 (WinRelRS6.050727-9100)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Drawing/2.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
WindowsLive.Writer.Localization
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLive.Writer.Localization.DLL
----------------------------------------
System.Windows.Forms
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.9149 (WinRelRS6.050727-9100)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Windows.Forms/2.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System.Configuration
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.9153 (WinRelRS6.050727-9100)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Configuration/2.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
----------------------------------------
System.Xml
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.9149 (WinRelRS6.050727-9100)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Xml/2.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------
WindowsLive.Writer.Api
Assembly Version: 1.1.0.0
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLive.Writer.Api.DLL
----------------------------------------
WindowsLiveLocal.WriterPlugin
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLiveLocal.WriterPlugin.DLL
----------------------------------------
WindowsLive.Writer.Controls
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLive.Writer.Controls.DLL
----------------------------------------
WindowsLive.Writer.Localization.resources
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/en/WindowsLive.Writer.Localization.resources.DLL
----------------------------------------
WindowsLive.Writer.HtmlParser
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLive.Writer.HtmlParser.DLL
----------------------------------------
WindowsLive.Writer.BlogClient
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLive.Writer.BlogClient.DLL
----------------------------------------
WindowsLive.Writer.Extensibility
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLive.Writer.Extensibility.DLL
----------------------------------------
WindowsLive.Writer.SpellChecker
Assembly Version: 16.4.3528.331
Win32 Version: 16.4.3528.331
CodeBase: file:///C:/Program%20Files%20(x86)/Windows%20Live/Writer/WindowsLive.Writer.SpellChecker.DLL
----------------------------------------
Accessibility
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.9149 (WinRelRS6.050727-9100)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/Accessibility/2.0.0.0__b03f5f7f11d50a3a/Accessibility.dll
----------------------------------------
Microsoft.mshtml
Assembly Version: 7.0.3300.0
Win32 Version: 7.0.3300.0
CodeBase: file:///C:/WINDOWS/assembly/GAC/Microsoft.mshtml/7.0.3300.0__b03f5f7f11d50a3a/Microsoft.mshtml.dll
----------------------------------------
CustomMarshalers
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.9149 (WinRelRS6.050727-9100)
CodeBase: file:///C:/WINDOWS/assembly/GAC_32/CustomMarshalers/2.0.0.0__b03f5f7f11d50a3a/CustomMarshalers.dll
----------------------------------------
System.Web
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.9154 (WinRelRS6.050727-9100)
CodeBase: file:///C:/WINDOWS/assembly/GAC_32/System.Web/2.0.0.0__b03f5f7f11d50a3a/System.Web.dll
----------------------------------------

************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.

For example:

<configuration>
<system.windows.forms jitDebugging="true" />
</configuration>

When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.

muta...@gmail.com

2021-03-29 11:01:27 UTC

Post by ***@gmail.com
I think in an aeroplane situation you need to be able
to recover from errors, but as a programmer, I'm far
more interested in isolating bugs than being nice to
users.

I looked up your MISRA C reference. I have no
confidence in people's ability to prevent bugs in
the first place, so I'll leave that to them to figure
out.

I'm more interested in what happens once I get
hold of their code.

I want to freeze the system to isolate the fault.
Every bug fixed makes the world a better place.

Post by Rod Pemberton
Halt switch. What if you had to do none of that because
everything worked perfectly? Is that a good scenario,
i.e., no headaches, or a bad scenario, i.e., loss of job
security?

I have no interest in job security. I try to make
myself redundant and dare them to sack me.

BFN. Paul.

Rod Pemberton

2021-03-30 08:54:26 UTC

On Mon, 29 Mar 2021 04:01:27 -0700 (PDT)

Post by ***@gmail.com
I think in an aeroplane situation you need to be able
to recover from errors, but as a programmer, I'm far
more interested in isolating bugs than being nice to
users.

I looked up your MISRA C reference. I have no
confidence in people's ability to prevent bugs in
the first place, so I'll leave that to them to figure
out.

There is actually a C which is mathematically proven to be correct.
Maybe, it's more to your liking. It's by Inria in France and is
named CompCert.

https://en.wikipedia.org/wiki/CompCert

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

James Harris

2021-03-31 09:23:30 UTC

On 26/03/2021 06:07, Rod Pemberton wrote:

...

Post by Rod Pemberton
application to terminate. It's interesting that engineers are taught
to NOT design in single points of failure, but programmers aren't.

That's interesting. How could one avoid single points of failure in a
program?

--
James Harris

Rod Pemberton

2021-04-01 11:59:52 UTC

On Wed, 31 Mar 2021 10:23:30 +0100

Post by Rod Pemberton
application to terminate. It's interesting that engineers are
taught to NOT design in single points of failure, but programmers
aren't.

That's interesting. How could one avoid single points of failure in a
program?

By doing everything possible to ensure that program execution continues
correctly without aborting or exiting due to errors or failures and
without introducing errors from continued program execution thereafter,
e.g., coding to avoid errors or failures, retrying after errors,
correction of detected errors, resetting of hardware, retrying after
failures instead of giving up, ignoring trivial errors which don't
affect the program's outcome, and not forcing automatic aborts and exits
upon all errors or all failures, etc.

My point really was a criticism of some programmers who intentionally
force programs to fail over any type of detected error, even if that
error is irrelevant to the proper execution of the program.

I'm sure many of the concepts of fault tolerance would apply.
https://en.wikipedia.org/wiki/Fault_tolerance
https://en.wikipedia.org/wiki/Robustness_(computer_science)
https://en.wikipedia.org/wiki/Defensive_programming

For databases, ACID and CRUD
https://en.wikipedia.org/wiki/ACID
https://en.wikipedia.org/wiki/Create,_read,_update_and_delete

For sequencing and/or control,
https://en.wikipedia.org/wiki/Lock_(computer_science)
https://en.wikipedia.org/wiki/Semaphore_(software)
https://en.wikipedia.org/wiki/Reference_counting
https://en.wikipedia.org/wiki/Go/no_go
https://en.wikipedia.org/wiki/Lockout–tagout

IIRC, in the past, we discussed the choice between potentially coding
an infinite loop versus coding a timeout for a loop which fails to
detect the expected I/O port data. A timeout would force a failure
because the loop doesn't properly complete (gives up without
completing), i.e., a single-point of failure. An infinite loop would
force a failure because the loop doesn't properly complete (keeps
retrying forever), i.e., a single-point of failure. An attempt to
avoid both single points of failure, the loop would likely need to
exit, the hardware would need to be reset and reprogrammed, and the
code would need to try the loop again, perhaps giving the user an
option of an exit if there is no resolution.

--

Rod Pemberton

2021-03-26 06:07:37 UTC

On Thu, 25 Mar 2021 03:20:19 -0700 (PDT)

Post by ***@gmail.com
I'm not sure where you got that idea from. I have two
tangible OSes at the moment, PDOS/386 and PDOS/3X0,
and it is true that both of them handle, to some extent,
executables + APIs from other OSes. But that was just
meant to be a starting point to begin to understand how
an OS works. It took me a long time to realize that a.out
executables could actually be relocated if they were
built properly. I instead had an elaborate system of
adjusting the base.
And my latest proposal is to replace all that previous
nonsense with a brand new OS that is totally portable,
and dependent only on a brand new BIOS, and that
that BIOS should have callback functions including
fopen()/fread() that access raw hard disks instead of
normal files that an OS deals with.

And, elsewhere in thread, you also said,

Post by ***@gmail.com
My primary interest is in
how to implement C90 to the maximum extent possible.

So, you're talking about,

a) portable OS only dependent on a new BIOS
b) implementing C90 to the maximum extent possible
c) (previously) about a user space OS which only
calls functions in the C library

You should know that C libraries are usually built up
from a small number of functions provided by the OS,
as it seems we discussed this previously for PDPCLIB.

As such, I don't understand why you need a new BIOS.
ISTM, that you only need a way to implement a handful
of system functions (which could use a BIOS but don't
have to ...), code a C90 library on top, code your
user space OS using that C library.

Recall that,

PDPCLIB calls 9 system functions:
read, write, open, close, lseek, rename, unlink, exit, and time

PDP-7/11 K&R C called 12 system functions:
open, close, read, write, creat, link, fork, exec, wait, exit, lseek,
pipe

Plauger's "Standard C Library" calls 18 system functions:
clock, close, environ, execl, exit, fork, getpid, kill, link, lseek,
open, read, sbrk, signal, time, unlink, wait, write

RedHat's newlib calls 19 system functions:
close, environ, _exit, execve, fork, fstat, getpid, isatty, kill, link,
lseek, open, read, sbrk, stat, times, unlink, wait, write

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

muta...@gmail.com

2021-03-26 22:13:52 UTC

Post by ***@gmail.com
And my latest proposal is to replace all that previous
nonsense with a brand new OS that is totally portable,
and dependent only on a brand new BIOS, and that
that BIOS should have callback functions including
fopen()/fread() that access raw hard disks instead of
normal files that an OS deals with.
My primary interest is in
how to implement C90 to the maximum extent possible.

a) and c) are the same, aren't they?

Post by Rod Pemberton
You should know that C libraries are usually built up
from a small number of functions provided by the OS,

Ok.

Post by Rod Pemberton
As such, I don't understand why you need a new BIOS.

How do you expect me to write any operating
system at all without a BIOS?

Assuming I do have a BIOS, why not define what
the BIOS should look like, instead of letting IBM
or whoever decide that?

Post by Rod Pemberton
ISTM, that you only need a way to implement a handful
of system functions (which could use a BIOS but don't
have to ...), code a C90 library on top, code your
user space OS using that C library.

I'm having trouble understanding your point.

1. My apps need a C library.
2. My OS needs a C library.
3. My BIOS can look like a reduced C library
4. With all that in place, things should be driven to
the max, including printing out the home address of
the sysadmin who put a quota restriction that caused
me to get a write error, and the exact date and time,
including UTC offset, of when he did that.

Post by Rod Pemberton
Recall that,
read, write, open, close, lseek, rename, unlink, exit, and time
open, close, read, write, creat, link, fork, exec, wait, exit, lseek,
pipe

Ok, interesting analysis, but I'm not sure what
your point is. :-)

Note that I checked, and you're talking about the
Linux 80386 target of PDPCLIB. That has fudged
memory allocation, so there should really be
another 2 syscalls if the Unix asshats had provided
a sys_malloc() and sys_free().

BFN. Paul.

Rod Pemberton

2021-03-27 00:06:19 UTC

On Fri, 26 Mar 2021 15:13:52 -0700 (PDT)

Post by Rod Pemberton
As such, I don't understand why you need a new BIOS.

How do you expect me to write any operating
system at all without a BIOS?

Program the hardware directly. But, all I meant here
was that the existing BIOS likely has the handful of
low level OS style functions you need.

Post by ***@gmail.com
Assuming I do have a BIOS, why not define what
the BIOS should look like, instead of letting IBM
or whoever decide that?

Are you going to create your own ROMs? Or, are you just
wrapping a different newer API over existing BIOSes?

Post by ***@gmail.com
I'm having trouble understanding your point.
1. My apps need a C library.
2. My OS needs a C library.
3. My BIOS can look like a reduced C library

Yes. Exactly. We're on the same page.

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

muta...@gmail.com

2021-03-27 10:53:11 UTC

Note to everyone, especially Scott - I'm not ignoring
your messages, I need time to get to them.

Post by Rod Pemberton
As such, I don't understand why you need a new BIOS.

How do you expect me to write any operating
system at all without a BIOS?

Program the hardware directly.

I need to get my program into memory before I can
do that, and I don't know how to do that without a BIOS.

On the mainframe I can do that (no choice currently).

I suppose the PC could be done the same way - start
banging I/O ports in the boot sector. Is that what you
had in mind?

Post by Rod Pemberton
But, all I meant here
was that the existing BIOS likely has the handful of
low level OS style functions you need.

Of course. And it's all tied to 8086 assembler
and one single machine type using 8086.

Post by ***@gmail.com
Assuming I do have a BIOS, why not define what
the BIOS should look like, instead of letting IBM
or whoever decide that?

Are you going to create your own ROMs? Or, are you just
wrapping a different newer API over existing BIOSes?

The latter for now. My BIOS, perhaps on top of
SudoBIOS, on top of IBM's (or whoever) BIOS.

Maybe one day it will be burnt in ROM. I don't care
at that level, so long as the interface is agreed on.

BFN. Paul.

wolfgang kern

2021-03-27 08:29:19 UTC

On 26.03.2021 23:13, ***@gmail.com wrote:
...

Post by ***@gmail.com
I'm having trouble understanding your point.
1. My apps need a C library.
2. My OS needs a C library.
3. My BIOS can look like a reduced C library
4. With all that in place, things should be driven to
the max, ...

To the MAX (applies to size only). Oh yes! that's what C is all about.

almost everything programmed with C become heavy bloat but it is Oh So
common and total convenient for all who are too lazy to use their brain.

the delivered BIOS is "the only" instance which knows about the hardware
it resides on. You may be able to emulate a few BIOS functions, but you
never know enough to emulate all the hardware.

So you cannot create a "portable BIOS".

I never used C nor any other HLL for my OS, so it's ~20 times faster
than comparable functions from windoze/loonix/etc. because it needs only
1/300 code+data size to do the same as the C-variants.

my C-lib favorite (disassembled):

Set_Carry
mov ebp,esp
push ebp
mov eax,0
sub eax,1 ;this really sets carry
pop ebp
mov esp,ebp
xor eax,eax ;but this clears it :)
ret

you'll find much more like this useless junk with a disassembler.
__
wolfgang

muta...@gmail.com

2021-03-27 11:06:55 UTC

To the MAX (applies to size only). Oh yes! that's what C is all about.

This is actually where I have remained ever since I
was armed with C90. Debating the last of the
assembler fanatics to try to get them to stand down.
Making sure my C code is producing acceptable
assembler code.

Post by wolfgang kern
almost everything programmed with C become heavy bloat but it is Oh So
common and total convenient for all who are too lazy to use their brain.

I am probably 20 times more productive in C90 than
I am in S/370 assembler or any other assembler.

In addition, modern C compilers produce ridiculously
good assembler. When I did write in assembler, I
wrote for ease of maintenance by another human.
C compilers don't care about that, and they will use
every trick imaginable. They therefore produce better
assembler than I would actually write.

And then, this ridiculously-good assembler is replicated
on a completely alien environment, simply by a recompile.
You need to rewrite yours from scratch, limiting the
number of machines you can support. I can instead
support every 16-bit, 32-bit and 64-bit platform known
to man, all at its natural word-size. Even specific
variations, such as 80386 vs 80486 can be targeted,
to make use of new instructions.

What is there not to love?

Do you have a specific complaint? You mentioned bloat -
why don't you personally just program in C but without
the bloat you dislike? Is there a technical barrier?

Post by wolfgang kern
the delivered BIOS is "the only" instance which knows about the hardware
it resides on. You may be able to emulate a few BIOS functions, but you
never know enough to emulate all the hardware.
So you cannot create a "portable BIOS".

The BIOS *interface* can be made portable, unless I'm
missing something. Obviously I expect the BIOS itself
to be rewritten for each piece of hardware, unless the
BIOS is actually a simple C program running under
Windows or whatever, using a file as a virtual hard
disk, and the OS has been carefully written so that it
can't even tell the difference. THAT is portable. ie a
"dummy BIOS" is portable.

Post by wolfgang kern
I never used C nor any other HLL for my OS, so it's ~20 times faster
than comparable functions from windoze/loonix/etc. because it needs only
1/300 code+data size to do the same as the C-variants.

When an application is running under an OS, 99.9%
of the CPU time should be spent on the application,
not the OS. So making the world's most efficient
(CPU-wise) OS is completely pointless. Or am I
missing something?

Post by wolfgang kern
Set_Carry
mov ebp,esp
push ebp
mov eax,0
sub eax,1 ;this really sets carry
pop ebp
mov esp,ebp
xor eax,eax ;but this clears it :)
ret
you'll find much more like this useless junk with a disassembler.

Perhaps change C library? PDPCLIB is public domain
so you are free to hack it to do whatever you want
with it, and slap any license you want on it.

BFN. Paul.

Bernhard Schornak

2021-03-27 16:20:59 UTC

Post by ***@gmail.com
In addition, modern C compilers produce ridiculously
good assembler.

Don't tell fairy tales:

http://st-intelligentdesign.blogspot.com/2010/04/13-appendix-2.html

GCC with *all* optimisations on and at maximum! I can post Megabyte
with weird code like that (I use AS as preferred assembler).

My assembler only libraries for 64 bit Windows:

https://drive.google.com/file/d/0B1OgMlxNnSNEVnJNeTNSeEM1bTQ/view?usp=sharing

Enjoy the weekend!

Bernhard Schornak

muta...@gmail.com

2021-03-28 12:08:13 UTC

Post by ***@gmail.com
In addition, modern C compilers produce ridiculously
good assembler.

GCC with *all* optimisations on and at maximum! I can post Megabyte
with weird code like that (I use AS as preferred assembler).

Ok, fine. I'm mainly talking about IBM's C compiler
for S/370 through to z/Arch.

That's the environment where I'm familiar with both
assembler and have inspected the assembler generated
by the C compiler. It is ridiculously efficient, even before
the compiler starts inlining functions.

You do need to take care to use memcpy() instead of
strcpy() to ensure you're matching the actual instruction
(MVC) available though.

BFN. Paul.

Bernhard Schornak

2021-03-28 14:26:00 UTC

Post by ***@gmail.com
You do need to take care to use memcpy() instead of
strcpy() to ensure you're matching the actual instruction
(MVC) available though.

Can't be more efficient than MOVDQA... ;) (My "strings" are aligned
to 16/32 byte boundaries to match the size of XMM/YMM registers.)

Greetings from Augsburg

Bernhard Schornak

muta...@gmail.com

2021-03-28 19:39:23 UTC

Post by ***@gmail.com
You do need to take care to use memcpy() instead of
strcpy() to ensure you're matching the actual instruction
(MVC) available though.

Can't be more efficient than MOVDQA... ;) (My "strings" are aligned
to 16/32 byte boundaries to match the size of XMM/YMM registers.)

Is there any reason why that technique couldn't
be added to a C compiler?

BFN. Paul.

Bernhard Schornak

2021-03-28 21:39:15 UTC

Post by ***@gmail.com
You do need to take care to use memcpy() instead of
strcpy() to ensure you're matching the actual instruction
(MVC) available though.

Can't be more efficient than MOVDQA... ;) (My "strings" are aligned
to 16/32 byte boundaries to match the size of XMM/YMM registers.)

Is there any reason why that technique couldn't
be added to a C compiler?

GCC is acknowledged to generate highly optimised code. But even
the best machine cannot beat the craftsmanship of manual coding
and hand-optimised code. A machine is good to create always the
same thing over and over again, with always the same precision,
but never will be good enough to create a program to do that on
its own. To do such a thing, we had to implement "human spirit"
into the program itself to enable it to invent genuine thoughts
and something like creativity. Even AI is far away from that...

Greetings from Augsburg

Bernhard Schornak

wolfgang kern

2021-03-29 04:59:00 UTC

Post by ***@gmail.com
You do need to take care to use memcpy() instead of
strcpy() to ensure you're matching the actual instruction
(MVC) available though.

Can't be more efficient than MOVDQA... ;) (My "strings" are aligned
to 16/32 byte boundaries to match the size of XMM/YMM registers.)

Is there any reason why that technique couldn't
be added to a C compiler?

Yeah! I work since many years on a telepathic transmitter ...
And even I may once succeed and add it to any compiler there wont
be better code to be seen. There aren't smart compilers available
and smart programmers are pretty rare to be found too.

Servas
__
wolfgang

muta...@gmail.com

2021-03-29 05:09:42 UTC

Post by wolfgang kern
And even I may once succeed and add it to any compiler there wont
be better code to be seen. There aren't smart compilers available
and smart programmers are pretty rare to be found too.

Are you using your super-optimized hand-written
code to code an operating system? If so, that's
pretty pointless, as almost 100% of the CPU time
is going to be spent in the application, not the
operating system. Even if the operating system
is written in Visual Basic it doesn't really matter,
speed-wise.

Or are you trying to squeeze your OS in to the
8086 (or 8080) memory limitations? If so, why
are you using 80386 instructions?

Also noting that if you stick to 80386 instructions
you can't take advantage of 80586 instructions.
And if you take advantage of 80586 instructions,
your OS will no longer run on 80386. You don't
have this problem if you write in C.

And regardless, even if you are writing an application
instead of an OS, do you actually have some
measurable cost savings that would justify coding
in assembler instead of C? ie do you make an EXTRA
profit of $1 million/year because you hand-crafted
assembler instead of C? If so, do you get even MORE
profit, e.g. $20 million/year, every time Intel puts out
a new processor, doubling the speed of your existing
code?

You mentioned "smart". Are you sure you're being
"smart"? I assume you have no interest in supporting
the Amiga or z/Arch platforms? Is there a reason for
that? Linux supports the latter, not sure about the
former.

BFN. Paul.

wolfgang kern

2021-03-29 09:40:33 UTC

Are you using your super-optimized hand-written
code to code an operating system?

Yes, I wrote during last forty years HEXWORK85 and KESYS95..18.

Post by ***@gmail.com
If so, that's
pretty pointless, as almost 100% of the CPU time
is going to be spent in the application, not the
operating system. Even if the operating system
is written in Visual Basic it doesn't really matter,
speed-wise.

then check on load time, my OS boots and deploy in 1.3 Seconds.

I also wrote all application for my OS.
(never ever execute foreign code, so KESYS is totally safe)

Post by ***@gmail.com
Or are you trying to squeeze your OS in to the
8086 (or 8080) memory limitations? If so, why
are you using 80386 instructions?

it started on an 80286 as a DOS extender and meanwhile grew up
to absolute standalone high security PM32 OS,

386(486++) instructions got more power than pure 8086 code.

Post by ***@gmail.com
Also noting that if you stick to 80386 instructions
you can't take advantage of 80586 instructions.
And if you take advantage of 80586 instructions,
your OS will no longer run on 80386. You don't
have this problem if you write in C.

I'm not stuck to 386 and I can use SSE2..5 instructions.

Post by ***@gmail.com
And regardless, even if you are writing an application
instead of an OS, do you actually have some
measurable cost savings that would justify coding
in assembler instead of C? ie do you make an EXTRA
profit of $1 million/year because you hand-crafted
assembler instead of C?

you may wonder how much faster than a C-coder I can finish any
programming task with my HEX-editable disassembler. It needs
much lesser keystrokes and I have unrestricted access to all
opcodes within all modes and never have weird syntax issues.

JFI: I use 32 event bits within one variable, C never heard
of bits and flags, so it makes true/false a 32 bit value.

Post by ***@gmail.com
If so, do you get even MORE
profit, e.g. $20 million/year, every time Intel puts out
a new processor, doubling the speed of your existing
code?

you're pretty wrong with such assumption, if a client asks
for hardware upgrade then I deliver with upgraded software.

Post by ***@gmail.com
You mentioned "smart". Are you sure you're being
"smart"? I assume you have no interest in supporting
the Amiga or z/Arch platforms? Is there a reason for
that? Linux supports the latter, not sure about the
former.

yeah I've been called smart/genius/weird and I'm not interested
to support Amiga nor any other museum devices.

why create an OS with C ?
it wont be any better than windoze or loonix, so save your time
and use them instead. They can emulate C64 and Amiga ...and DOS.

you mean C-source is portable to any other family ? sure possible,
but the produced bloatware may not fit, not in size nor in speed.
__
wolfgang

muta...@gmail.com

2021-03-29 11:11:26 UTC

Post by wolfgang kern
then check on load time, my OS boots and deploy in 1.3 Seconds.

And approximately how many times per day does
an end user boot your system?

And how long do they spend running applications
after booting?

Post by ***@gmail.com
If so, do you get even MORE
profit, e.g. $20 million/year, every time Intel puts out
a new processor, doubling the speed of your existing
code?

you're pretty wrong with such assumption, if a client asks
for hardware upgrade then I deliver with upgraded software.

You have the ability to double the efficiency of
your own code?

yeah I've been called smart/genius/weird and I'm not interested
to support Amiga nor any other museum devices.

Ok, well I don't like to tie myself down to any particular
computer. I know very little about the Raspberry Pi or
the ARM instruction set, but when I arrive there, I expect
to arrive with a shitload of tools that work out of the box.

Post by wolfgang kern
why create an OS with C ?

Portable and public domain.

Post by wolfgang kern
it wont be any better than windoze or loonix, so save your time
and use them instead. They can emulate C64 and Amiga ...and DOS.

Not interested in other people's copyrighted software.

Post by wolfgang kern
you mean C-source is portable to any other family ? sure possible,
but the produced bloatware may not fit, not in size nor in speed.

True. My C library when built for the 68000 is 39k in size.
I'm probably going to have trouble when I arrive at the
Commodore 64. Nevermind 16k systems that CP/M
supported. I'm hoping that I can at least make it to the
Commodore 128 with the OS and C library sitting in the
top 64k.

But yeah, I don't know what to do on a 16k system or
even a 64k system. So far I'm still fighting the 8086.
Got to stop the hardcoding of the number "4". And
see if the MSDOS executable format needs to change.
And see if .com files can be relocatable instead of
requiring a 64k boundary.

BFN. Paul.

wolfgang kern

2021-03-29 12:44:56 UTC

Post by wolfgang kern
then check on load time, my OS boots and deploy in 1.3 Seconds.

And approximately how many times per day does
an end user boot your system?
And how long do they spend running applications
after booting?

this was just an speed compare example, all other things like additional
functions and apps are short and fast and load extremely fast as well.

Post by ***@gmail.com
If so, do you get even MORE
profit, e.g. $20 million/year, every time Intel puts out
a new processor, doubling the speed of your existing
code?

you're pretty wrong with such assumption, if a client asks
for hardware upgrade then I deliver with upgraded software.

You have the ability to double the efficiency of
your own code?

not sure you mean here, my code is already optimized for a given
hardware, it will speed up when I get faster CPU and faster RAM,
and it would take me only minutes to alter a few required bytes.

Post by wolfgang kern
why create an OS with C ?

Portable and public domain.

some believe in gods,
many in gold,
a few in logic and
C-coders in the portable compiler lie.

my OS and all code around it isn't public domain.
I had to feed my kids with the earnings.

Post by wolfgang kern
you mean C-source is portable to any other family ? sure possible,
but the produced bloatware may not fit, not in size nor in speed.

True. My C library when built for the 68000 is 39k in size.
I'm probably going to have trouble when I arrive at the
Commodore 64. Nevermind 16k systems that CP/M
supported. I'm hoping that I can at least make it to the
Commodore 128 with the OS and C library sitting in the
top 64k.
But yeah, I don't know what to do on a 16k system or
even a 64k system. So far I'm still fighting the 8086.
Got to stop the hardcoding of the number "4". And
see if the MSDOS executable format needs to change.

you mean the additional four bits to achieve 1MB range?
why not use UnReal mode and flat addressing ?

Post by ***@gmail.com
And see if .com files can be relocatable instead of
requiring a 64k boundary.
64K .com files ? make them .bin and load with INT0x13

__
wolfgang

muta...@gmail.com

2021-03-29 14:08:26 UTC

Post by wolfgang kern
then check on load time, my OS boots and deploy in 1.3 Seconds.

And approximately how many times per day does
an end user boot your system?
And how long do they spend running applications
after booting?

this was just an speed compare example, all other things like additional
functions and apps are short and fast and load extremely fast as well.

Speeding up an app makes sense. Not the OS.

Post by ***@gmail.com
If so, do you get even MORE
profit, e.g. $20 million/year, every time Intel puts out
a new processor, doubling the speed of your existing
code?

you're pretty wrong with such assumption, if a client asks
for hardware upgrade then I deliver with upgraded software.

You have the ability to double the efficiency of
your own code?

not sure you mean here, my code is already optimized for a given
hardware, it will speed up when I get faster CPU and faster RAM,
and it would take me only minutes to alter a few required bytes.

For some reason you think there is an importance in
a particular bit of code being fast. Intel will actually
do that for you without you having to do anything at
all. When Intel doubles the speed of your program,
does that translate into a profit for someone? Who
and how much?

Post by wolfgang kern
why create an OS with C ?

Portable and public domain.

some believe in gods,
many in gold,
a few in logic and
C-coders in the portable compiler lie.

It is portable. Which bit are you claiming is not portable?
You can start with the fat.c code. It handles FAT-12,
FAT-16, FAT-32 and long filenames. I can't see any
barrier to making that work on S/3X0 EBCDIC, although I
haven't done that yet.

Post by wolfgang kern
my OS and all code around it isn't public domain.
I had to feed my kids with the earnings.

Sure. And that is why you didn't use Linux and
Windows either.

There are multiple reasons for developing an OS,
and I gave you mine.

I am questioning the need to write it in assembler
instead of C. Not to not make it public domain.

Post by wolfgang kern
you mean C-source is portable to any other family ? sure possible,
but the produced bloatware may not fit, not in size nor in speed.

True. My C library when built for the 68000 is 39k in size.
I'm probably going to have trouble when I arrive at the
Commodore 64. Nevermind 16k systems that CP/M
supported. I'm hoping that I can at least make it to the
Commodore 128 with the OS and C library sitting in the
top 64k.
But yeah, I don't know what to do on a 16k system or
even a 64k system. So far I'm still fighting the 8086.
Got to stop the hardcoding of the number "4". And
see if the MSDOS executable format needs to change.

you mean the additional four bits to achieve 1MB range?

Yes. I want that number "4" to be flexible in all software.
So that new hardware, such as 8086+, with a 16-bit
segment shift, suddenly gives access to 4 GiB.

Post by wolfgang kern
why not use UnReal mode and flat addressing ?

I expect existing 8086 software to work with the
new address range, with no change to anything
at all except a new CPU.

Post by ***@gmail.com
And see if .com files can be relocatable instead of
requiring a 64k boundary.
64K .com files ? make them .bin and load with INT0x13

No. I want a 1k CP/M program to be located at address
0x2000:0x1000

And another 1k CP/M program to be located at address
0x2000:0x2000

Instead of everyone demanding a 64k boundary.

That way when the segment shift becomes 16 instead
of 4, even that space isn't wasted.

BFN. Paul.

wolfgang kern

2021-03-30 12:51:05 UTC

Post by wolfgang kern
not sure you mean here, my code is already optimized for a given
hardware, it will speed up when I get faster CPU and faster RAM,
and it would take me only minutes to alter a few required bytes.

For some reason you think there is an importance in
a particular bit of code being fast.

No, I make _all_ code short and fast.

Post by ***@gmail.com
Intel will actually
do that for you without you having to do anything at
all. When Intel doubles the speed of your program,
does that translate into a profit for someone? Who
and how much?

??? how could Intel change the speed on my machines ???
[240 PCs sold with KESYS]

Post by wolfgang kern
why create an OS with C ?

Portable and public domain.

some believe in gods,
many in gold,
a few in logic and
C-coders in the portable compiler lie.

and now try to make it work for the KESYS-filesystem :)

Post by ***@gmail.com
There are multiple reasons for developing an OS,
and I gave you mine.

the main reason to create my own were freedom and independence.
and the lack on proper tools made me make own toolset as well.

Post by ***@gmail.com
I am questioning the need to write it in assembler
instead of C. Not to not make it public domain.

no bloat, no restrictions, no weird syntax issues, no need
for preserving registers, full bit/flag support ... and
not at least I'm not bound to all those shit in libraries.

BUT, I don't use any assembler, I write pure metal in hex
and I use my brain as the only trustworthy compiler.

Post by wolfgang kern
you mean the additional four bits to achieve 1MB range?

Yes. I want that number "4" to be flexible in all software.
So that new hardware, such as 8086+, with a 16-bit
segment shift, suddenly gives access to 4 GiB.

Post by wolfgang kern
why not use UnReal mode and flat addressing ?

I expect existing 8086 software to work with the
new address range, with no change to anything
at all except a new CPU.

existing software doesn't know about your idea :)

I see, you became an CPU architect. hope you got enough coins,
because this your new CPU will cost an arm and a leg :)
__
wolfgang

muta...@gmail.com

2021-03-30 18:19:29 UTC

Post by ***@gmail.com
For some reason you think there is an importance in
a particular bit of code being fast.

No, I make _all_ code short and fast.

Which is completely pointless if you are writing
a "hello, world" program that is going to be
executed once. If you are wasting developer
resources and forgoing maintainability just so
you can save 1 clock cycle in world history,
what's the point?

Are you expecting your children to maintain
your KESYS?

??? how could Intel change the speed on my machines ???
[240 PCs sold with KESYS]

And what happens when one of those PCs is stolen?
Is there a replacement? What is the CPU speed of
the replacement PC compared to the old one? Is it
5 times faster? Does the 5 times faster PC benefit
the user, generating massive profits, or anything at
all? If so, maybe the user should having a standing
order with a local thief to regularly have his PC
stolen so that he can make huge profits from the
5 times faster PC delivered by you/Intel.

Post by wolfgang kern
why create an OS with C ?

Portable and public domain.

some believe in gods,
many in gold,
a few in logic and
C-coders in the portable compiler lie.

and now try to make it work for the KESYS-filesystem :)

I'm pretty sure I could write a KESYS file system handler
that will work on every environment known to man, so
long as there are sufficient resources, e.g. an 8086.

Post by ***@gmail.com
There are multiple reasons for developing an OS,
and I gave you mine.

the main reason to create my own were freedom and independence.
and the lack on proper tools made me make own toolset as well.

What's wrong with the tools that I used to create
PDOS? They're clearly good enough to create an OS.

Post by ***@gmail.com
I am questioning the need to write it in assembler
instead of C. Not to not make it public domain.

no bloat,

Unless you're trying to squeeze this in to 16k (which
I admit I don't know how to do with PDOS), I don't
know why you care.

Post by wolfgang kern
no restrictions,

There are C compilers with source, so you can
change the language if you don't like it. Is there
something specific you don't like about the C
language that would make you tie yourself down
to one single processor?

Post by wolfgang kern
no weird syntax issues,

Out of all the languages in the world, 80386
machine code had the best syntax?

Post by wolfgang kern
no need for preserving registers,

If you write in C, you don't need to even be aware
of what the registers are called on ARM or whatever.

Post by wolfgang kern
full bit/flag support ... and

I have written a C library and an OS for multiple
platforms and have never noticed a problem with
bits/flags. Perhaps you could show me something
specific in PDOS where it prints "2 + 3 = 7" or
something because of the lack of bit/flag support.

Post by wolfgang kern
not at least I'm not bound to all those shit in libraries.

The source for PDPCLIB is public domain. You can
delete/change any part of that you don't like, and you
can slap your own copyright on the derived work so
that you can continue to feed your family.

If you don't like PDPCLIB, there is also PDCLIB which
is also public domain.

Post by wolfgang kern
BUT, I don't use any assembler, I write pure metal in hex
and I use my brain as the only trustworthy compiler.

And overly-burdened whoever has to maintain your code.
C programmers on the other hand are a dime a dozen.

Post by wolfgang kern
you mean the additional four bits to achieve 1MB range?

Yes. I want that number "4" to be flexible in all software.
So that new hardware, such as 8086+, with a 16-bit
segment shift, suddenly gives access to 4 GiB.

Post by wolfgang kern
why not use UnReal mode and flat addressing ?

I expect existing 8086 software to work with the
new address range, with no change to anything
at all except a new CPU.

existing software doesn't know about your idea :)

Existing software shouldn't have been written to care
about my idea. The code generated by a C compiler,
in any memory model, does not have the number "4"
hardcoded. So what's the barrier?

Post by wolfgang kern
I see, you became an CPU architect. hope you got enough coins,
because this your new CPU will cost an arm and a leg :)

It is enough to get it working in Bochs. Then you can
do "mem" or whatever under PDOS/86 or maybe
Freedos and you will be surprised to see "3.9 GB free".

BFN. Paul.

wolfgang kern

2021-03-31 11:54:26 UTC

Post by ***@gmail.com
For some reason you think there is an importance in
a particular bit of code being fast.

No, I make _all_ code short and fast.

my OS isn't made for hello world programs, the main
core is 512KB and the standard modules span 2560KB.
Then comes the client ordered stuff, usually <4MB.

so the whole memory image after deploy is max. 8MB.
and now compare it to other OS memory images which
have full GUI support and its own filesystem.
KESYS modules imply a scientific 256 bit calculator,
support user scripts (aka token/parameter listings),
LAN connection, multiple mice, changeable keyboard
layouts, 16 fonts and up to three language sets.
But it doesn't have internet (as safety demanded).

Post by ***@gmail.com
Are you expecting your children to maintain
your KESYS?

No, even my kids know a lot of my work and could help
they have other professions and what's maintain ?
my sold units work flawless and don't need frequent
upgrades like M$ or Mozilla weekly needed bug-fixes.

when a client asked for new hardware or software add-ons
then I delivered it. of course not FOC :)

??? how could Intel change the speed on my machines ???
[240 PCs sold with KESYS]

And what happens when one of those PCs is stolen?
Is there a replacement? What is the CPU speed of
the replacement PC compared to the old one?

this actually happened once. the client got an equal
replacement from me (insurance paid for it).

Post by ***@gmail.com
Is it 5 times faster? Does the 5 times faster PC benefit
the user, generating massive profits, or anything at
all? If so, maybe the user should having a standing
order with a local thief to regularly have his PC
stolen so that he can make huge profits from the
5 times faster PC delivered by you/Intel.

what weird ideas, you don't know much about business ?

Post by wolfgang kern
and now try to make it work for the KESYS-filesystem :)

I'm pretty sure I could write a KESYS file system handler
that will work on every environment known to man, so
long as there are sufficient resources, e.g. an 8086.

total wrong assumptions, C cannot mix modes nor work bit wise.

Post by ***@gmail.com
There are multiple reasons for developing an OS,
and I gave you mine.

the main reason to create my own were freedom and independence.
and the lack on proper tools made me make own toolset as well.

What's wrong with the tools that I used to create
PDOS? They're clearly good enough to create an OS.

it's your OS and your decision. I just wonder why reinvent what's
here since several decades.

Post by ***@gmail.com
I am questioning the need to write it in assembler
instead of C. Not to not make it public domain.

no bloat,

Unless you're trying to squeeze this in to 16k (which
I admit I don't know how to do with PDOS), I don't
know why you care.

Post by wolfgang kern
no restrictions,

Yes! working for one specific CPU made my solutions best.

You still believe in portability ? you can't even recompile
8086 source into Long Mode code. the intrinsic for AAM09==??

Post by wolfgang kern
no weird syntax issues,

Out of all the languages in the world, 80386
machine code had the best syntax?

machine code doesn't have a syntax at all.

Post by wolfgang kern
no need for preserving registers,

If you write in C, you don't need to even be aware
of what the registers are called on ARM or whatever.

I can use all available registers w/o restricions.

Post by wolfgang kern
full bit/flag support ... and

OK just one out of thousands: "classical semaphore"

CHECK_GRANT_HLL:
mov ecx,false ;assuming true means 'free' yet.
mov eax,true
lock cmpxchg,dw global_grant_if_free, ecx
;eax true means granted and modified to false
;eax false means occupied
ret ;also possible but not supported: jz granted
___________
CHECK_GRANT_ASM:
mov eax,grant_if_free_bitnr
lock bts dw global_flags,eax ;can span a range +/-256MByte
;a set CY means the flag was SET and remains SET(occupied)
;NC means granted and is SET now
jnc granted ;also possible but not recommended: ret

can you see the difference ?
and now check what your beloved C would produce for such.

And one more from my OS:
IDLE:
mov ecx,0x10
STI
cmp [event],0 ;16bits for 16 timers, 16 for hardware
;set by IRQ-routines,cleared by OS
jz idle
mov eax,[event]
BTC eax,ecx ;start with hardware
INC ecx
jc do_17 ;## from 17..32 (RTC,SPK,HD,SND,MSY/KBD,..)
BTC eax,ecx
INC ecx
jc do_18

yes it's spaghetti, but shorter and faster than a branch LUT.

Post by wolfgang kern
BUT, I don't use any assembler, I write pure metal in hex
and I use my brain as the only trustworthy compiler.

And overly-burdened whoever has to maintain your code.
C programmers on the other hand are a dime a dozen.

Again, there is nothing to maintain. I need to alter just a
few bytes for hardware changes.
__
wolfgang

Rod Pemberton

2021-04-01 11:47:31 UTC

On Wed, 31 Mar 2021 13:54:26 +0200

Post by wolfgang kern
C cannot mix modes

True.

Post by wolfgang kern
C cannot [...] work bit wise.

False. C has numerous bit-wise operators.

Post by wolfgang kern
machine code doesn't have a syntax at all.

Machine code has a binary format which if not complied with will cause
the machine code to fail at run-time. This is really no different of a
concept than a syntax error for a high-level language which causes the
code to fail at compile-time (or input-time for interpreted languages).
For the earlier x86 opcodes, this machine code format is 2-3-3 octal.

--

wolfgang kern

2021-04-01 14:12:18 UTC

Post by Rod Pemberton
On Wed, 31 Mar 2021 13:54:26 +0200

Post by wolfgang kern
C cannot mix modes

True.

Post by wolfgang kern
C cannot [...] work bit wise.

False. C has numerous bit-wise operators.

OK, but I never saw any BT(CSR_) instruction used in C-compiled code,
what I found instead were AND/OR combos.
and you sure break your nails when try to modify cc-flags.

Post by wolfgang kern
machine code doesn't have a syntax at all.

:) sure true except hexadecimal notation doesn't follow any other rule
than 0..9 A..F and a few cursor-keys. [no dots 2dots nor underline].
knowledge of instructions and its format variations is of course vital.

OK I cheat a bit, because my tool immediate disassembles my hex input
so I see if what I do makes any sense or not.
__
wolfgang

Rod Pemberton

2021-03-30 08:55:40 UTC

On Mon, 29 Mar 2021 14:44:56 +0200

some believe in [...] and
C-coders in the portable compiler lie.

Hey, you know I don't believe that.

C is 33% portable, 33% compatible, 33% non-portable.

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

Rod Pemberton

2021-03-30 08:56:00 UTC

On Mon, 29 Mar 2021 04:11:26 -0700 (PDT)

You have the ability to double the efficiency of your own code?

No man has the ability to out think themselves
(for non-trivial problems).

When a man's most superior strategy fails as the result of an even more
superior strategy by someone else, that man has lost forever.

Obviously, in order to best the other person's strategy, the man must
develop a strategy that is superior to the other person's strategy.
But, to do that, the man must also concoct a strategy that is far
superior to his own previously superior strategy. This is an impossible
task as a person is only so smart or intelligent. In other words, to
out think yourself, your I.Q. must be increasing. Unfortunately, your
I.Q. peaks around age 20 years and declines thereafter. Hence, you
can't out think yourself. Someone else can out think you, if they're
smarter or have a higher I.Q. In other words, your peak intellect is
capped, which you used fully to develop your most superior strategy that
was defeated. So, if you concocted the absolute best strategy, the most
superior strategy that you could for your given I.Q., then you can go
no further.

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

Rod Pemberton

2021-03-30 09:09:30 UTC

On Mon, 29 Mar 2021 04:11:26 -0700 (PDT)

Post by wolfgang kern
why create an OS with C ?

Portable and public domain.

Don't forget flexibility and rapid development.

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

Bernhard Schornak

2021-03-29 21:22:01 UTC

Post by ***@gmail.com
You do need to take care to use memcpy() instead of
strcpy() to ensure you're matching the actual instruction
(MVC) available though.

Can't be more efficient than MOVDQA... ;) (My "strings" are aligned
to 16/32 byte boundaries to match the size of XMM/YMM registers.)

Is there any reason why that technique couldn't
be added to a C compiler?

Yep! Just let the poor C disciple spend his life in darkness - no one
should be forced to see farther than he can fumble around himself...

Pfüat'Di

Bernhard

Rod Pemberton

2021-03-30 08:55:25 UTC

On Sun, 28 Mar 2021 23:39:15 +0200

Post by Bernhard Schornak
GCC is acknowledged to generate highly optimised code.

It comes nowhere close to that produced by OpenWatcom.

Post by Bernhard Schornak
But even the best machine cannot beat the craftsmanship of manual
coding and hand-optimised code.

Obviously, you and wolfgang are much better than most at assembly. If
the two of you learned C, I have no doubt that you'd both be much
better than most at coding C too.

So, for specific situations, which aren't suitable for a
general-purpose language like C, e.g., graphics, parallelism, vectors,
etc, I agree that assembly coded by an "expert" can outperform a C
compiler's optimizations. However, I disagree when it comes to
general-purpose programming by ordinary programmers, e.g., most
applications. For general-purpose code, C compilers produce much
better code than the ordinary people who attempt to code assembly and
do so poorly, who also attempt to code C and do so somewhat adequately.

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

Scott Lurndal

2021-03-30 14:15:58 UTC

Post by Rod Pemberton
On Sun, 28 Mar 2021 23:39:15 +0200

Post by Bernhard Schornak
GCC is acknowledged to generate highly optimised code.

It comes nowhere close to that produced by OpenWatcom.

Post by Bernhard Schornak
But even the best machine cannot beat the craftsmanship of manual
coding and hand-optimised code.

Obviously, you and wolfgang are much better than most at assembly. If
the two of you learned C, I have no doubt that you'd both be much
better than most at coding C too.

It may be worth pointing out that the glibc implementations of
memcpy and memset are hand-coded assembly....

Bernhard Schornak

2021-03-30 23:14:33 UTC

Post by Rod Pemberton
On Sun, 28 Mar 2021 23:39:15 +0200

Post by Bernhard Schornak
GCC is acknowledged to generate highly optimised code.

It comes nowhere close to that produced by OpenWatcom.

Post by Bernhard Schornak
But even the best machine cannot beat the craftsmanship of manual
coding and hand-optimised code.

Obviously, you and wolfgang are much better than most at assembly. If
the two of you learned C, I have no doubt that you'd both be much
better than most at coding C too.
So, for specific situations, which aren't suitable for a
general-purpose language like C, e.g., graphics, parallelism, vectors,
etc, I agree that assembly coded by an "expert" can outperform a C
compiler's optimizations. However, I disagree when it comes to
general-purpose programming by ordinary programmers, e.g., most
applications. For general-purpose code, C compilers produce much
better code than the ordinary people who attempt to code assembly and
do so poorly, who also attempt to code C and do so somewhat adequately.

The last sentence is more or less a "confession of failure" for High
Level Languages per se, isn't it? ;)

I tried to code in "C" the first years I switched to OS/2, but there
were too much restrictions and the language is really hard to under-
stand if you're used to code in assembler. After some time to get in
touch with your instruction set, coding in assembler is much faster,
because you can do what you want on a straight forward road, while a
"C" compiler needs all that casting and other weird things which are
not related to the things you really want to do. It's like wearing a
too small corset constricting each move you want to make. (Which, by
the way, is the reason, why the output of *any* compiler cannot beat
hand optimised code - neither in complex tasks nor in trivial code.)

The only advantage of High Level and Scripting Languages is, that it
is faster to write halfways working code and let the consumer do the
testing. If you take a look back in time, machines are some thousand
times faster than they were in the 1990'ies, but performing the same
tasks still requires the same time. While machines are a top product
of engineering skills, the code controlling the machines became much
larger and behaves like thick molasses, trying to thwart the machine
down, rather than to make use of the new features and get it running
at top speed...

Greetings from Augsburg

Bernhard Schornak

muta...@gmail.com

2021-03-31 02:34:41 UTC

Post by Bernhard Schornak
I tried to code in "C" the first years I switched to OS/2, but there
were too much restrictions and the language is really hard to under-
stand if you're used to code in assembler. After some time to get in
touch with your instruction set, coding in assembler is much faster,
because you can do what you want on a straight forward road, while a
"C" compiler needs all that casting and other weird things which are
not related to the things you really want to do. It's like wearing a
too small corset constricting each move you want to make. (Which, by

I'm probably 20 times more productive in C than
assembler. Even if I have to occasionally use a
cast. I'm happy to help you with your C if you want.

Some S/370 assembler guy from France was asking
where he could learn C in Paris and I told him if he
liked, I could try teaching him C via email, for free,
and see if it worked out. It worked out fine. He was
extremely impressed with what his C compiler was
able to do.

I'm not some pro-French bigot, so if you'd like me to
teach you C on whatever platform you are using, I'm
happy to help you for free too. As Rod noted, if you're
going to look at the generated assembler, install
Watcom C rather than GCC if you don't want to be
disappointed.

Post by Bernhard Schornak
the way, is the reason, why the output of *any* compiler cannot beat
hand optimised code - neither in complex tasks nor in trivial code.)

That is technically true at some level, but not true
in practice for most people.

Post by Bernhard Schornak
The only advantage of High Level and Scripting Languages is, that it
is faster to write halfways working code and let the consumer do the
testing. If you take a look back in time, machines are some thousand
times faster than they were in the 1990'ies, but performing the same
tasks still requires the same time. While machines are a top product
of engineering skills, the code controlling the machines became much
larger and behaves like thick molasses, trying to thwart the machine
down, rather than to make use of the new features and get it running
at top speed...

I can agree with that. My C code is not like that. My OS
plus an app on 8086 is about 150k. That's the price you
need to pay for the features C90 gives you. I can probably
reduce it a bit by effectively only having one copy of the
C library. The size doesn't change much when I rebuild
for an 80386 instead. Or a 68020. Or S/3X0.

BFN. Paul.

Bernhard Schornak

2021-03-31 05:11:00 UTC

I'm probably 20 times more productive in C than
assembler. Even if I have to occasionally use a
cast. I'm happy to help you with your C if you want.
Some S/370 assembler guy from France was asking
where he could learn C in Paris and I told him if he
liked, I could try teaching him C via email, for free,
and see if it worked out. It worked out fine. He was
extremely impressed with what his C compiler was
able to do.
I'm not some pro-French bigot, so if you'd like me to
teach you C on whatever platform you are using, I'm
happy to help you for free too. As Rod noted, if you're
going to look at the generated assembler, install
Watcom C rather than GCC if you don't want to be
disappointed.

I doubt that Watcom C is able to compile *.S files written AT&T syntax.
*All* of my code (~30 MB) is written in AT&T syntax...

Post by Bernhard Schornak
the way, is the reason, why the output of *any* compiler cannot beat
hand optimised code - neither in complex tasks nor in trivial code.)

That is technically true at some level, but not true
in practice for most people.

Have a look at this

https://drive.google.com/file/d/0B1OgMlxNnSNEbXI3VHdYMFN0aWM/view?usp=sharing

the source code and resources are included in the zip.

Greetings from Augsburg

Bernhard Schornak

muta...@gmail.com

2021-03-31 06:42:24 UTC

Post by ***@gmail.com
I'm not some pro-French bigot, so if you'd like me to
teach you C on whatever platform you are using, I'm
happy to help you for free too. As Rod noted, if you're
going to look at the generated assembler, install
Watcom C rather than GCC if you don't want to be
disappointed.

I doubt that Watcom C is able to compile *.S files written AT&T syntax.
*All* of my code (~30 MB) is written in AT&T syntax...

I'm not sure if we are talking cross-purposes.
Watcom C is for compiling C code if you wish
to try coding in C and wish to look at the
generated assembler to make sure you are
happy the compiler did a good job.

Post by ***@gmail.com
I can agree with that. My C code is not like that. My OS
plus an app on 8086 is about 150k. That's the price you
need to pay for the features C90 gives you. I can probably
reduce it a bit by effectively only having one copy of the
C library. The size doesn't change much when I rebuild
for an 80386 instead. Or a 68020. Or S/3X0.

Have a look at this
https://drive.google.com/file/d/0B1OgMlxNnSNEbXI3VHdYMFN0aWM/view?usp=sharing
the source code and resources are included in the zip.

I don't know why you gave me that. I took a look and
all I saw was code that has no chance of running even
on an 80386, nevermind z/Arch, nevermind ARM, and
nevermind whatever they invent tomorrow.

Not a single portable C file.

BFN. Paul.

Bernhard Schornak

2021-03-31 07:36:02 UTC

I doubt that Watcom C is able to compile *.S files written AT&T syntax.
*All* of my code (~30 MB) is written in AT&T syntax...

I'm not sure if we are talking cross-purposes.

No. I always was talking about x86-64 assembler, or, for OS/2,
x86-32.

Post by ***@gmail.com
Watcom C is for compiling C code if you wish
to try coding in C and wish to look at the
generated assembler to make sure you are
happy the compiler did a good job.

Yep. I thought so. I don't understand why you suggest me a "C"
compiler, while I definitely told you that I use GCC's "AS" as
my default assembler and "_make.cmd" as replacement for "Make"
(which is not part of the MinGW package).

Have a look at this
https://drive.google.com/file/d/0B1OgMlxNnSNEbXI3VHdYMFN0aWM/view?usp=sharing
the source code and resources are included in the zip.

I don't know why you gave me that. I took a look and
all I saw was code that has no chance of running even
on an 80386, nevermind z/Arch, nevermind ARM, and
nevermind whatever they invent tomorrow.

The OS/2 versions (up to 7.2.) were x86-32, but OS/2 denies to
run on modern hardware. This year is 2021, not 1983, where all
PCs ran DOS (or one of the DOS clones). Today, most people run
64 bit Windoze on their machines - no need to stay with 32 bit
(or less...) technology, which is outdated since more than one
decade.

Post by ***@gmail.com
Not a single portable C file.

Yep - no unneccessary instructions to remove and no "molasses"
slowing down processing. The more remarkable thing is, that my
programs are multi-lingual. You can switch to another language
in the blink of an eye (program => language). The main windows
of my programs are all independent of the screen's resolution,
and always pop up at the same spot and with the same width and
height ratio. Not to forget that built-in database engine, the
most important part of my programs, where all data are stored.
Together with the functions of this program itself - a latency
test for 4 testees where results are shown *immediately* after
the test run plus an evaluation dialog - this program occupies
mere 67 KByte. Try to beat that with the HLL of your choice...

Greetings from Augsburg

Bernhard Schornak

muta...@gmail.com

2021-03-31 09:48:22 UTC

Post by ***@gmail.com
Watcom C is for compiling C code if you wish
to try coding in C and wish to look at the
generated assembler to make sure you are
happy the compiler did a good job.

Yep. I thought so. I don't understand why you suggest me a "C"
compiler,

I thought you were complaining about GCC's
generated assembler, and Rod recommended
Watcom if that was a concern.

Post by Bernhard Schornak
The OS/2 versions (up to 7.2.) were x86-32, but OS/2 denies to
run on modern hardware. This year is 2021, not 1983, where all
PCs ran DOS (or one of the DOS clones). Today, most people run
64 bit Windoze on their machines - no need to stay with 32 bit
(or less...) technology, which is outdated since more than one
decade.

No reason not to support the old OS. Some people
still run it. It costs nothing at all to support it if you
write your program in portable C90.

Post by Bernhard Schornak
Yep - no unneccessary instructions to remove and no "molasses"
slowing down processing. The more remarkable thing is, that my
programs are multi-lingual. You can switch to another language
in the blink of an eye (program => language). The main windows
of my programs are all independent of the screen's resolution,
and always pop up at the same spot and with the same width and
height ratio. Not to forget that built-in database engine, the
most important part of my programs, where all data are stored.
Together with the functions of this program itself - a latency
test for 4 testees where results are shown *immediately* after
the test run plus an evaluation dialog - this program occupies

You can do all the above in C90 and STILL support OS/2,
and ARM etc etc, for ZERO extra cost. Quite apart from
the fact that no-one is paying you a bonus to display a
window in 0.01 seconds instead of 0.02 seconds.

Post by Bernhard Schornak
mere 67 KByte. Try to beat that with the HLL of your choice...

The 64-bit machines you target (and are stuck with), can
address 16 EiB.

Are you waiting until we have 128-bit computers before
you are happy to increase your 67 KB program to 200K?
If someone was paying you to restrict your program
to 67 KB it would be for an environment like the
Commodore 128 where you don't have much choice.
But you are far away from supporting that platform.
I'm not.

Quite apart from the fact that the overhead of my C90
library (PDPCLIB) is only about 35k.

BFN. Paul.

Bernhard Schornak

2021-03-31 13:39:43 UTC

Post by ***@gmail.com
Watcom C is for compiling C code if you wish
to try coding in C and wish to look at the
generated assembler to make sure you are
happy the compiler did a good job.

Yep. I thought so. I don't understand why you suggest me a "C"
compiler,

I thought you were complaining about GCC's
generated assembler, and Rod recommended
Watcom if that was a concern.

Please read that conversation, again.

No reason not to support the old OS. Some people
still run it. It costs nothing at all to support it if you
write your program in portable C90.

I don't write programs in "portable" languages. My platform is
x86-64, only. Some few tasks for my Arduino/ESP8266/ESP32 toys
are C, 'cause I don't want to learn multiple instruction sets,
but the quality of that C code I write doesn't matter in terms
of execution time - most of the code are waiting loops used as
timers to trigger events or read sensors periodically.

Please read, what others write - I explicitly told you, that I
had to stop my OS/2 development, because OS/2 denied to run on
the latest AMD processor generation called "Bulldozer" in 2009
(which was the reason why *many* people stopped using OS/2).

Post by Bernhard Schornak
mere 67 KByte. Try to beat that with the HLL of your choice...

The 64-bit machines you target (and are stuck with), can
address 16 EiB.

"Long ModeThis mode is unique to the AMD64 architecture. This
mode supports up to 4 petabytes of physical-address space using
52-bit physical addresses. Long mode requires the use of page-
translation and the physical-address size extensions (PAE)."

AMD Programmer's Manual Volume 2 (24593.pdf), 1.1.2, page 4

No, But it could run 32 tasks simultaneously, even if I do not
*need* more than one core. If you write code that occupies the
entire machine, because it assumes, it always will be the only
thing running on that machine, then you probably need to learn
how modern operating systems work - all running programs share
the available cores, and there are *hundreds* of tasks waiting
to be executed.

Post by ***@gmail.com
Are you waiting until we have 128-bit computers before
you are happy to increase your 67 KB program to 200K?
If someone was paying you to restrict your program
to 67 KB it would be for an environment like the
Commodore 128 where you don't have much choice.
But you are far away from supporting that platform.
I'm not.
Quite apart from the fact that the overhead of my C90
library (PDPCLIB) is only about 35k.

I think, it is pointless to discuss this further - there is no
way to find a consensus in any point.

Greetings from Augsburg

Bernhard Schornak

Rod Pemberton

2021-03-31 09:09:05 UTC

On Wed, 31 Mar 2021 01:14:33 +0200

Post by Rod Pemberton
On Sun, 28 Mar 2021 23:39:15 +0200

Post by Bernhard Schornak
But even the best machine cannot beat the craftsmanship of manual
coding and hand-optimised code.

Obviously, you and wolfgang are much better than most at assembly.
If the two of you learned C, I have no doubt that you'd both be much
better than most at coding C too.
So, for specific situations, which aren't suitable for a
general-purpose language like C, e.g., graphics, parallelism,
vectors, etc, I agree that assembly coded by an "expert" can
outperform a C compiler's optimizations. However, I disagree when
it comes to general-purpose programming by ordinary programmers,
e.g., most applications. For general-purpose code, C compilers
produce much better code than the ordinary people who attempt to
code assembly and do so poorly, who also attempt to code C and do
so somewhat adequately.

The last sentence is more or less a "confession of failure" for High
Level Languages per se, isn't it? ;)

No, I don't see it that way. I see it as a "confession of failure" for
assembly, a partial failure for HLLs, but not because of the languages.
I think programming attracts too many people who were gamers and want
to code games. So, programming doesn't attract enough people who are
smart, unlike professions such as engineering or physics or mathematics
which attract plenty of smart people.

C has plenty of dark corners which need to be avoided. Some of these
are flaws in the language, or bizarre interactions, or holdovers from
language changes. If you stick to the main elements of C and avoid the
problematic areas, you'll do much better with it. I actually program
in a subset of C to avoid problematic areas. For my OS code, I also
restrict the C library usage to stdio.h, stdlib.h, and string.h. For
bootstrapping code, you can restrict it to just stdio.h. Of course, a
C novice won't be able to do that, i.e., will need to use much or all
of the C library. You shouldn't restrict library usage for normal C
applications.

As mentioned elsewhere, I've coded in a number of mostly older HLLs
(Logo, BASIC, Pascal, FORTRAN, C, PL/1, Forth), and also 6502 and x86
assembly (basic 16-bit/32-bit and 32-bit PM setup, no FPU/SSE/SIMD, no
64-bit, etc). I've done a tiny bit of Javascript and html programming,
and I just had to fix some Python code a day or two ago. Of the HLLs,
the only two that I really liked were BASIC and C. The older variant of
PL/1 I coded in was as powerful as C, but was more like Pascal with
pointers and counted strings. In PL/1, everything passed-by-reference
by default, which was a **MAJOR** improvement over C. There were only
two pass-by-values in over 5MLOC of code for a brokerage. However, I
didn't like the language because it was too Pascal like, was much more
verbose, and so needed more typing. The syntax of C is much more
balanced too.

Post by Bernhard Schornak
After some time to get in
touch with your instruction set, coding in assembler is much faster,
because you can do what you want on a straight forward road, while a
"C" compiler needs all that casting and other weird things which are
not related to the things you really want to do.

With assembly you must keep track of variables and flags on your own.
Is the variable in memory, on the stack, in a register? Is there an
intermediate instruction which destroys my flag? I was quite good at
keeping flag values alive with 6502, but it's much more work with x86.

--

Bernhard Schornak

2021-03-31 13:37:37 UTC

Post by Rod Pemberton
On Wed, 31 Mar 2021 01:14:33 +0200

Post by Rod Pemberton
On Sun, 28 Mar 2021 23:39:15 +0200

Post by Bernhard Schornak
But even the best machine cannot beat the craftsmanship of manual
coding and hand-optimised code.

Obviously, you and wolfgang are much better than most at assembly.
If the two of you learned C, I have no doubt that you'd both be much
better than most at coding C too.
So, for specific situations, which aren't suitable for a
general-purpose language like C, e.g., graphics, parallelism,
vectors, etc, I agree that assembly coded by an "expert" can
outperform a C compiler's optimizations. However, I disagree when
it comes to general-purpose programming by ordinary programmers,
e.g., most applications. For general-purpose code, C compilers
produce much better code than the ordinary people who attempt to
code assembly and do so poorly, who also attempt to code C and do
so somewhat adequately.

The last sentence is more or less a "confession of failure" for High
Level Languages per se, isn't it? ;)

If I remember right, one of the first "non-professional" games was
Doom - a highly optimised program, entirely coded in assembler. It
was State Of The Art in 3-D graphics.

C has plenty of dark corners which need to be avoided. Some of these
are flaws in the language, or bizarre interactions, or holdovers from
language changes. If you stick to the main elements of C and avoid the
problematic areas, you'll do much better with it. I actually program
in a subset of C to avoid problematic areas. For my OS code, I also
restrict the C library usage to stdio.h, stdlib.h, and string.h. For
bootstrapping code, you can restrict it to just stdio.h. Of course, a
C novice won't be able to do that, i.e., will need to use much or all
of the C library. You shouldn't restrict library usage for normal C
applications.
As mentioned elsewhere, I've coded in a number of mostly older HLLs
(Logo, BASIC, Pascal, FORTRAN, C, PL/1, Forth), and also 6502 and x86
assembly (basic 16-bit/32-bit and 32-bit PM setup, no FPU/SSE/SIMD, no
64-bit, etc). I've done a tiny bit of Javascript and html programming,
and I just had to fix some Python code a day or two ago. Of the HLLs,
the only two that I really liked were BASIC and C. The older variant of
PL/1 I coded in was as powerful as C, but was more like Pascal with
pointers and counted strings. In PL/1, everything passed-by-reference
by default, which was a **MAJOR** improvement over C. There were only
two pass-by-values in over 5MLOC of code for a brokerage. However, I
didn't like the language because it was too Pascal like, was much more
verbose, and so needed more typing. The syntax of C is much more
balanced too.

If it is about scripting, HTML with CSS were my favourite choice -
an easy to use, easy to understand language you can learn by doing
(my former homepage had more than 800 pages completely linked with
the others). PHP is easy to learn, too, but all of the other lang-
uages are ways too complicated to learn them without a teacher.

It's just a question of how many registers are available. The more
registers, the less you need to access "slow" memory. Most reg-reg
operations are executed in a single clock cycle, and, depending on
the processor's internal execution pipes, several instructions can
be executed simultaneously (if fed in the proper order). In 32-bit
code, you are restricted to 7 registers, because it was a *really*
bad idea to use SP/ESP/RSP as general purpose register. In 64-bit,
you get 8 additional registers, which should be enough for most of
our daily tasks (sometimes I wish there were one or two more...).

You should know your instruction set, of course, but that is like
learning the elements of a higher level language. What you get is
full control over the hardware - as far as the OS lets you... ;)

Greetings from Augsburg

Bernhard Schornak

Rod Pemberton

2021-04-01 11:46:27 UTC

On Wed, 31 Mar 2021 15:37:37 +0200

Post by Bernhard Schornak
If I remember right, one of the first "non-professional" games was
Doom - a highly optimised program, entirely coded in assembler. It
was State Of The Art in 3-D graphics.

Sorry, you don't remember right. Doom was ANSI C.
https://en.wikipedia.org/wiki/Development_of_Doom

John Carmack is a C programmer. E.g., he developed QuakeC fnr Quake,
coded Quake in C for DJGPP (GCC) which produced DPMI exe's for MS-DOS,
and he used the LCC C compiler for Quake3.

--

Rod Pemberton

2021-04-01 12:02:18 UTC

On Thu, 1 Apr 2021 06:46:27 -0500

Post by Rod Pemberton
On Wed, 31 Mar 2021 15:37:37 +0200

Post by Bernhard Schornak
If I remember right, one of the first "non-professional" games was
Doom - a highly optimised program, entirely coded in assembler. It
was State Of The Art in 3-D graphics.

Sorry, you don't remember right. Doom was ANSI C.
https://en.wikipedia.org/wiki/Development_of_Doom
John Carmack is a C programmer. E.g., he developed QuakeC fnr Quake,
coded Quake in C for DJGPP (GCC) which produced DPMI exe's for MS-DOS,
and he used the LCC C compiler for Quake3.

You're probably remembering one of Carmack's optimized functions:
https://en.wikipedia.org/wiki/Defensive_programming

--

Bernhard Schornak

2021-04-01 16:30:20 UTC

Post by Rod Pemberton
On Wed, 31 Mar 2021 15:37:37 +0200

Post by Bernhard Schornak
If I remember right, one of the first "non-professional" games was
Doom - a highly optimised program, entirely coded in assembler. It
was State Of The Art in 3-D graphics.

Sorry, you don't remember right. Doom was ANSI C.
https://en.wikipedia.org/wiki/Development_of_Doom
John Carmack is a C programmer. E.g., he developed QuakeC fnr Quake,
coded Quake in C for DJGPP (GCC) which produced DPMI exe's for MS-DOS,
and he used the LCC C compiler for Quake3.

QuakeC was 1996, Doom 1993 and Wolfenstein 3D 1992. I don't like Ego-
shooters, so I never played those games, and only know of them 'cause
I almost learned the content of computer magazines by heart. That was
my only source of information. After some digging:

https://github.com/id-Software/DOOM/tree/master/linuxdoom-1.10

At the bottom you find the assembly part of that code. Not that much,
as I thought, but probably one of the reasons why everybody mentioned
the extraordinary speed of the graphics. So I correct my claim to "It
was partially written in assembly!".

Teaches me that I shouldn't trust everything I read in magazines. ;)

Greetings from Augsburg

Bernhard Schornak

Rod Pemberton

2021-03-27 12:27:05 UTC

On Sat, 27 Mar 2021 09:29:19 +0100

Post by wolfgang kern
almost everything programmed with C become heavy bloat but it is Oh So
common and total convenient for all who are too lazy to use their brain.

That's an interesting comment.

It's not interesting comment because it's easier to code in C than
assembly (it is IMO), but because many people attempting to program in
C say that C is a difficult programming language to learn and use.

I guess people just want to be really, really lazy, nearly brain dead.

Having coded lots of 6502, a bit of x86, tons of C, lots of PL/1, a bit
of FORTRAN, a bit of Pascal, plenty of BASIC, I'm still a believer that
C is the best programming language. Of course, I haven't learned LISP,
COBOL, SNOBOL, or any modern languages like Perl, PHP, Python, Rust,
Ruby, R, Swift, ... There are now a bunch of no-code languages. These
are also known as visual app builders:

https://www.appian.com/
https://www.appsheet.com/
https://www.appypie.com/
https://www.bettyblocks.com/
https://botsify.com/
https://www.budibase.com/
https://chatfuel.com/
https://debuild.co/
https://draftbit.com/
https://www.kgbase.com/
https://www.outsystems.com/
https://www.skuid.com/
https://www.unqork.com/

Post by wolfgang kern
Set_Carry
mov ebp,esp
push ebp
mov eax,0
sub eax,1 ;this really sets carry
pop ebp
mov esp,ebp
xor eax,eax ;but this clears it :)
ret

Is this hand coded? ...

a) If so, it appears to me that someone switched the xor and mov 0 by
mistake. mov is usually used to preserve flags, which might be
needed for sbb but not for sub.

b) If not, my guess is this is the result of inlined assembly in C that
was compiled. Someone thought they could set the carry flag with
inlined assembly "mov eax, 0; sub eax, 1", but didn't realize that the C
compiler was using both eax and the carry flag. These are referred
to as in-use registers (and flags), and a programmer needs to know
which ones are usable or must be preserved, i.e., not clobbered. So,
when the compiler sets the return value for the C function, it clears
both eax and the carry flag, causing the function to fail to do what the
programmer expected. (I'm assuming that compiled inlined assembly in C
is the actual situation at hand.)

Which C library?

FYI, the C language doesn't use a carry. Unfortunately, it's a feature
that C lacks. If a carry flag was present in C, small integers could
be combined or chained into larger integers. So, if C doesn't have a
carry (or carry flag), why is this function within a C library? I.e.,
setting the carry flag is necessary to call an x86 BIOS function.
That's why.

The ebp/esp code is just prolog and epilog code for the C function
subroutine, which sets up the function parameters on the stack, and
returns them to the calling C function, as well as creates a stack frame
for the procedure. In this case, the ebp/esp code is unnecessary,
because nothing is accessed on the stack, and (in theory) this code
should be optimized away.

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

Rod Pemberton

2021-03-27 12:34:33 UTC

On Sat, 27 Mar 2021 07:27:05 -0500

Post by Rod Pemberton
b) If not, my guess is this is the result of inlined assembly in C
that was compiled. Someone thought they could set the carry flag with
inlined assembly "mov eax, 0; sub eax, 1", but didn't realize that
the C compiler was using both eax and the carry flag. These are
referred to as in-use registers (and flags), and a programmer needs
to know which ones are usable or must be preserved, i.e., not
clobbered. So, when the compiler sets the return value for the C
function, it clears both eax and the carry flag, causing the function
to fail to do what the programmer expected. (I'm assuming that
compiled inlined assembly in C is the actual situation at hand.)

Some C compilers allow you to declare functions to be "naked" so they
don't generate the prolog/epilog code. That would've fixed the
problem here, by removing the ebp/esp code. So, it's possible a
compiler specific function attribute was removed or forgotten.

--
Countries that won't talk to Biden: North Korea, China, Russia, Iran.

wolfgang kern

2021-03-27 16:49:04 UTC

On 27.03.2021 13:34, Rod Pemberton wrote:
...

Post by Rod Pemberton
clobbered. So, when the compiler sets the return value for the C
function, it clears both eax and the carry flag, causing the function
to fail to do what the programmer expected. (I'm assuming that
compiled inlined assembly in C is the actual situation at hand.)

my "set_carry" is only one byte large: 0xF9.

I disassembled lots (>50GB) of DLLs and applications during last decades.
it shows almost everywhere never used stack frame copies and redundant
compiler tombstones.

Post by Rod Pemberton
Some C compilers allow you to declare functions to be "naked" so they
don't generate the prolog/epilog code. That would've fixed the
problem here, by removing the ebp/esp code. So, it's possible a
compiler specific function attribute was removed or forgotten.

sure possible but rare seen it used that way.

And what I dislike most on any HLL is the calling convention. I use
functionl parameters by reg val or reg ptr whatsoever fit best but
really rare (only in one function) over the stack.
__
wolfgang

muta...@gmail.com

2021-03-28 12:10:15 UTC

Post by wolfgang kern
And what I dislike most on any HLL is the calling convention. I use
functionl parameters by reg val or reg ptr whatsoever fit best but
really rare (only in one function) over the stack.

Just change to Watcom C and you can get parameters
passed by register. If that is even required, because I
assume it will inline functions in the first place. All a
completely unmanageable mess if you try to do the
same thing in assembler.

BFN. Paul.

Bernhard Schornak

2021-03-27 16:20:32 UTC

The typical C brainf*ck at its best! God save the stack frame!

Pfüat'Di!

Bernhard

James Harris

2021-03-31 09:21:45 UTC

Post by James Harris
As this group is about OS development my reply was about what I believe
an OS or run-time system /ought/ to do

And that is *exactly* what I am interested in, even if
I *personally* don't have the resources to implement it.
I'd like to at least *understand* how things *should*
work.

Cool.

...

Do you consider this to be a serious shortcoming in C90
that should be rectified? If so, is it possible to rectify it
in a portable manner?

AISI whether to include or omit automatic exception handling is simply a
choice. Some languages have it, some do not. With C90 you can use either
setjmp/longjmp or explicit code to handle exceptions but if you want
setjmp and longjmp to be portable then they are very limited.

Perhaps you could add automatic exception handling to C90 but then it
would not be C90.

Post by James Harris
programmer has to check for exceptions manually, usually along the lines of
retcode = syscall1();
if (retcode != OK)
{
/* Handle the exception */
}

Is there a problem doing it that way? That's the only way
I've ever coded it. I've never moved beyond C90 so I am
not attached to any alternate paradigm. I wanted to
exhaust C90 first.

There are two problems with that:

1. The exception-handling code bulks up and obscures the mainline code,
making the logic of the program harder to work with.

2. If a programmer fails to include all necessary code to handle
exceptions then they can go undetected.

I don't know why you would say that. Why can't this
be done in the language in a standard manner?

That's because the language won't have the details needed. Consider this
potential flow of calls

program --> language runtime --> OS library --> OS kernel --> driver

If the driver at the lowest leve detects an exception it has all the
info it could possibly want about what went wrong, and could package
that info up into a neat notification. The kernel, library and language
runtime could maybe add extra information as the exception notification
is passed back up and then the program would have full info as to what
went wrong.

But if the OS and its components do not provide the notification -
perhaps returning a generic "IO error" - then by the time the
notification gets to the language runtime it will have very little info
as to what went wrong.

Hence you'd need to get the underlying OS to provide all the relevant
information.

Post by ***@gmail.com
Along the lines of char *get_last_error(); which
potentially returns a 10,000-byte string full of
information that can be dumped to stderr. With the
C90+ standard giving the implementation the
option of returning just "\n" to every call.

If we are talking about how things /should/ be done it's not as simple
as that. For example, one exception can lead to another so by the time a
program gets back control there could be multiple exceptions which have
been raised but not yet handled.

Post by ***@gmail.com
Are you perhaps drawing a distinction between
"language" and "OS" that I'm not seeing? Given that
all of C's functionality is in functions, it seems to me
that it is actually impossible to differentiate between
the language and the OS. Is there a tangible difference
between fopen/open/DosOpen?

If you are saying that the traditional boundaries of language and OS
should be removed then I agree with you.

--
James Harris

muta...@gmail.com

2021-03-31 09:35:26 UTC

Post by ***@gmail.com
Do you consider this to be a serious shortcoming in C90
that should be rectified? If so, is it possible to rectify it
in a portable manner?

AISI whether to include or omit automatic exception handling is simply a
choice. Some languages have it, some do not. With C90 you can use either
setjmp/longjmp or explicit code to handle exceptions but if you want
setjmp and longjmp to be portable then they are very limited.
Perhaps you could add automatic exception handling to C90 but then it
would not be C90.

I'm happy to create a new language. It would be good
if it was based on C90 and the new features were
largely "implementation-defined" so that on a
Commodore 64 they can all be #define'd to nothing.

Post by James Harris
1. The exception-handling code bulks up and obscures the mainline code,
making the logic of the program harder to work with.
2. If a programmer fails to include all necessary code to handle
exceptions then they can go undetected.

I see.

I don't know why you would say that. Why can't this
be done in the language in a standard manner?

That's because the language won't have the details needed. Consider this
potential flow of calls
program --> language runtime --> OS library --> OS kernel --> driver
If the driver at the lowest leve detects an exception it has all the
info it could possibly want about what went wrong, and could package
that info up into a neat notification. The kernel, library and language
runtime could maybe add extra information as the exception notification
is passed back up and then the program would have full info as to what
went wrong.
But if the OS and its components do not provide the notification -
perhaps returning a generic "IO error" - then by the time the
notification gets to the language runtime it will have very little info
as to what went wrong.
Hence you'd need to get the underlying OS to provide all the relevant
information.

Ok, this is alt.os.development. We have control of the OS.
I would argue the OS should log all the above information
independently of the application.

And then pass the same information up to the application.

How about compiling all programs with a parameter
"stop dead on exception"?

I refused to implement abend handlers on MVS too.
And signals. Whenever a bug occurs, I want everything
to stop dead.