Discussion:
UEFI boot basics
(too old to reply)
James Harris
2023-11-24 17:59:47 UTC
Permalink
Here are some notes on getting UEFI to boot one's own code.

To be clear about where the code below is intended to fit into an OS
boot process the idea is to boot the same OS by various means such as,
in simple terms,

BIOS --> my BIOS bootloader \
PXE --> my PXE bootloader } my OS
UEFI --> my UEFI bootloader /

and this thread is intended to be about the UEFI bootloader part.

Furthermore, while I can add some comments later about a development
environment and accessing a boot filesystem but this post is just about
getting started with a UEFI Hello World. There's already plenty in here
to make the post over-long as it is.

There are specs for UEFI (and ACPI) at

https://uefi.org/specifications

I'll refer to sections of what is currently the latest spec, i.e. 2.10.

UEFI documents are not easy reading. The basic spec, alone, is something
like 2,000 pages and obscure.

To make things worse, I've seen tutorials which add further libraries.
Such libraries are intended 'to make things easier' but they give a
programmer more to learn and make it hard to tell what is UEFI and what
is an added library. I wanted to find out about the fundamentals of UEFI
itself so what I'll describe is a relatively low-level approach.

Perhaps the best place to start is the video at



and other videos by the same (very patient!) content creator.

At least in my test environment I found that even persuading UEFI to run
code was a major undertaking. For too long the boot process led to
nothing happening at all. No output, no error message, no response of
any kind. For a long while I couldn't tell whether my code had even been
booted. So getting to the point where one can output text is crucial:
once one can write text to the display then one has a chance of
developing and debugging.

With that said here's UEFI code (in Nasm assembly syntax) to write a
Hello message to the screen. I'll walk through it, below.

efi_main is:



;Called with x64 ms abi
; RCX = image
; RDX = systab

global efi_main
efi_main:

push rbp
mov rbp, rsp
sub rsp, 32
mov r10, rdi ;image
mov r11, rsi ;systab

lea rdx, [rel msg_hello]
mov rcx, [r11 + efi_systab.conout]
mov rax, [rcx + efi_stop.OutputString]
call rax

add rsp, 32
pop rbp
ret

section .data

msg_hello: dw __?utf16?__("Hello 64 UEFI..."), 13, 10, 0



The first thing to say is that the design of UEFI appears to be heavily
influenced by Microsoft. As such, the string to be printed needs CR as
well as LF to go to the start of a new line. Also, the file which we
want UEFI to boot needs to be in PE format. There's no option of flat
binary or ELF etc.

However, once one's own bootloader is running it can load the OS from
whatever type of file one wants. In other words, the OS loader has to be
in PE format but what it loads can be of some other format as required.

When UEFI calls our code and when we call UEFI routines we naturally
have to stick to the expected calling conventions. They have to match
the CPU and mode. For 64-bit x86 booting see section 2.3.4. which is
about x64 Platforms.

The spec says that a caller must always call with the stack 16-byte
aligned. One thing I haven't found out is whether that means the stack
should be 16-byte aligned before or after the call instruction.
(Naturally, the call instruction will add 8 bytes to the stack.) The
above code assumes 'before'. So pushing RBP will realign the stack. But
that needs to be revisited.

One thing UEFI gives the booted code is a pointer to the System Table.
The table is documented in section 4. It starts with the following
fields. (I included only as far as I needed.)

struc efi_systab ;UEFI system table
.hdr: resb efi_hdr_size
.fvend: resq 1 ;FirmwareVendor
.frev: resd 1 ;FirmwareRevision
resd 1 ;padding
.coninh: resq 1 ;ConsoleInHandle
.conin: resq 1 ;ConIn
.conouth: resq 1 ;ConsoleOutHandle
.conout: resq 1 ;ConOut
endstruc

The header is


struc efi_hdr ;Generic header
.Signature: resq 1
.Revision: resd 1
.HeaderSize: resd 1
.CRC32: resd 1
resd 1 ;reserved
endstruc

A note for anyone not familiar with C: where the spec in section 4.3.1 says

CHAR16 *FirmwareVendor;
UINT32 FirmwareRevision;

whereas FirmwareRevision in the second line is uint32, as you might
expect, the asterisk on the first line indicates that FirmwareVendor is
a /pointer/ to the stated type, char16, and so in this case the field is
64-bit.

Going back to the code, the System Table's conout field is a pointer to
a 'simple text output' (STO) table for the console, i.e. the screen at
boot time. An STO table includes the following fields.

struc efi_stop ;UEFI simple text output protocol
.Reset: resq 1
.OutputString: resq 1
.TestString: resq 1
.QueryMode: resq 1
.SetMode: resq 1
.SetAttribute: resq 1
.ClearScreen: resq 1
.SetCursorPosition: resq 1
.EnableCursor: resq 1
.Mode: resq 1
endstruc

In that, the OutputString field holds the address of the UEFI function
which can be called to write a string of 16-bit chars to the screen. It
is passed

ECX = the STO table for the console (i.e. conout)
EDX = the string to print

Lo and behold, text appears! At least it does if one has everything
right. I can say more about that if anyone wants but for now I'll stop
here and not make this post any longer.

Feel free to add comments, questions and corrections.
--
James Harris
wolfgang kern
2023-11-25 06:13:53 UTC
Permalink
Post by James Harris
;Called with x64 ms abi
;   RCX = image
;   RDX = systab
where to ??? address of this call ms abi is
[rcx + efi_stop.OutputString] ???
Post by James Harris
global efi_main
  push rbp
  mov rbp, rsp
  sub rsp, 32
  mov r10, rdi ;image
  mov r11, rsi ;systab
  lea rdx, [rel msg_hello]
  mov rcx, [r11 + efi_systab.conout]
  mov rax, [rcx + efi_stop.OutputString]
  call rax
  add rsp, 32
  pop rbp
  ret
section .data
msg_hello: dw __?utf16?__("Hello 64 UEFI..."), 13, 10, 0
...
Post by James Harris
So pushing RBP will realign the stack. But that needs to be revisited.
yes, I'd try stack aligning from very start w/o using RBP at all,

may the call return any useful parameter on the stack or in REGs ?
or can we just ignore all of it.
Post by James Harris
struc efi_stop ;UEFI simple text output protocol
.Reset: resq 1
.OutputString: resq 1
.TestString: resq 1
.QueryMode: resq 1
.SetMode: resq 1
.SetAttribute: resq 1
.ClearScreen: resq 1
.SetCursorPosition: resq 1
.EnableCursor: resq 1
.Mode: resq 1
endstruc
how would this struct look in my notes: ?
I assume resq 1 mean 64 bit pointers

000 qq reset
008 qq OutputString
010 qq ...
...
048 qq mode

and how can we know the value for the call ?
is there a thunk table somewhere or windoze styled call by "name".
Post by James Harris
In that, the OutputString field holds the address of the UEFI function
THAT'S the question: how to find these function addresses ?
Post by James Harris
which can be called to write a string of 16-bit chars to the screen.
It is passed
ECX = the STO table for the console (i.e. conout)
EDX = the string to print
One thing UEFI gives the booted code is a pointer to the System Table.
The table is documented in section 4. It starts with the following
fields. (I included only as far as I needed.)
how. where and when will UEFI give me that?
Post by James Harris
struc efi_systab ;UEFI system table
  .hdr:      resb efi_hdr_size
  .fvend:    resq 1 ;FirmwareVendor
  .frev:     resd 1 ;FirmwareRevision
             resd 1 ;padding
  .coninh:   resq 1 ;ConsoleInHandle
  .conin:    resq 1 ;ConIn
  .conouth:  resq 1 ;ConsoleOutHandle
  .conout:   resq 1 ;ConOut
endstruc
The header is
struc efi_hdr ;Generic header
  .Signature:    resq 1
  .Revision:     resd 1
  .HeaderSize:   resd 1
  .CRC32:        resd 1
                 resd 1  ;reserved
endstruc
;can you please show me the above again in my notation
don't mind I already got your type cast style :)
Post by James Harris
A note for anyone not familiar with C: where the spec in section 4.3.1 says
  CHAR16                           *FirmwareVendor;
  UINT32                           FirmwareRevision;
whereas FirmwareRevision in the second line is uint32, as you might
expect, the asterisk on the first line indicates that FirmwareVendor is
a /pointer/ to the stated type, char16, and so in this case the field is
64-bit.
now I'm totally confused yet, CHAR16 means 64 bit ?
it seems it's just another 64 bit pointer.
Post by James Harris
Going back to the code, the System Table's conout field is a pointer to
a 'simple text output' (STO) table for the console, i.e. the screen at
boot time. An STO table includes the following fields.
struc efi_stop ;UEFI simple text output protocol
  .Reset:                   resq 1
  .OutputString:            resq 1
  .TestString:              resq 1
  .QueryMode:               resq 1
  .SetMode:                 resq 1
  .SetAttribute:            resq 1
  .ClearScreen:             resq 1
  .SetCursorPosition:       resq 1
  .EnableCursor:            resq 1
  .Mode:                    resq 1
endstruc
In that, the OutputString field holds the address of the UEFI function
which can be called to write a string of 16-bit chars to the screen. It
is passed
  ECX = the STO table for the console (i.e. conout)
  EDX = the string to print
Lo and behold, text appears! At least it does if one has everything
right. I can say more about that if anyone wants but for now I'll stop
here and not make this post any longer.
Feel free to add comments, questions and corrections.
thanks a lot, I now learned how to interpret your RESb/d/q :)
__
wolfgang
James Harris
2023-11-25 13:18:30 UTC
Permalink
Post by James Harris
;Called with x64 ms abi
;   RCX = image
;   RDX = systab
where to ???  address of this call ms abi is
[rcx + efi_stop.OutputString] ???
No, take a look at the spec's section 2.3.4.1. Handoff State. It shows
that when UEFI calls an application it passes in the following.

Rcx - EFI_HANDLE
Rdx - EFI_SYSTEM_TABLE*
RSP - <return address>

IOW it passes us two parameters: an efi handle and something it calls
EFI_SYSTEM_TABLE*. The asterisk is C speak for 'pointer' so that second
parameter in RDX is a pointer to the efi system table.

The final part of the handoff state appears to mean only that RSP will
point to the return address.
Post by James Harris
global efi_main
   push rbp
   mov rbp, rsp
   sub rsp, 32
   mov r10, rdi ;image
   mov r11, rsi ;systab
   lea rdx, [rel msg_hello]
   mov rcx, [r11 + efi_systab.conout]
   mov rax, [rcx + efi_stop.OutputString]
   call rax
   add rsp, 32
   pop rbp
   ret
section .data
msg_hello: dw __?utf16?__("Hello 64 UEFI..."), 13, 10, 0
...
Post by James Harris
So pushing RBP will realign the stack. But that needs to be revisited.
yes, I'd try stack aligning from very start w/o using RBP at all,
may the call return any useful parameter on the stack or in REGs ?
or can we just ignore all of it.
Yes, have a read through section 2.3.4.2. Detailed Calling Conventions.
It includes "Return values that fix [which I guess is supposed to be
"fit"] into 64-bits are returned in the Rax register" etc.
Post by James Harris
struc efi_stop ;UEFI simple text output protocol
    .Reset:                   resq 1
    .OutputString:            resq 1
    .TestString:              resq 1
    .QueryMode:               resq 1
    .SetMode:                 resq 1
    .SetAttribute:            resq 1
    .ClearScreen:             resq 1
    .SetCursorPosition:       resq 1
    .EnableCursor:            resq 1
    .Mode:                    resq 1
endstruc
how would this struct look in my notes: ?
I assume resq 1 mean 64 bit pointers
Not quite. All that that code is doing is reserving space, not saying
what should be in that space (pointer, integer, float etc). Nasm strucs
are documented in

https://www.nasm.us/xdoc/2.16.01/html/nasmdoc5.html#section-5.9.1

but perhaps more useful would be the sizes:

resb - reserve 1 byte
resw - reserve 1 word (2 bytes as far as Nasm is concerned)
resd - reserve 1 doubleword (4 bytes in Nasm)
resq - reserve 1 quadword (8 bytes in Nasm)
000 qq reset
008 qq OutputString
010 qq ...
...
048 qq mode
and how can we know the value for the call ?
is there a thunk table somewhere or windoze styled call by "name".
I don't know what Windows does but since posting yesterday I realised
that I should have changed my code a bit (added better comments and used
some different registers) and I had missed off code which you would need
which I didn't write but which is brought in by the link step. I'll show
the final executable's full disassembly below.

The new 'application' code is

efi_main:
push r13
push r12
push rbp
mov rbp, rsp
sub rsp, 32

;Save the supplied parameters in caller-save registers
mov r12, rdi ;image
mov r13, rsi ;systab

;Set up parms for text output
lea rdx, [rel msg_hello]
mov rcx, [r13 + efi_systab.conout]

;Make the call
mov rax, [rcx + efi_sto.OutputString]
call rax

add rsp, 32
pop rbp
pop r12
pop r13
ret

and the full disassembly which includes the above code and also the key
_start entry point is

$ objdump bin/x64cpu/bootx64b.efi -d -Mintel

bin/x64cpu/bootx64b.efi: file format pei-x86-64


Disassembly of section .text:

0000000000002000 <_text>:
2000: 41 55 push r13
2002: 41 54 push r12
2004: 55 push rbp
2005: 48 89 e5 mov rbp,rsp
2008: 48 83 ec 20 sub rsp,0x20
200c: 49 89 fc mov r12,rdi
200f: 49 89 f5 mov r13,rsi
2012: 48 8d 15 e7 1f 00 00 lea rdx,[rip+0x1fe7] # 4000
<_data>
2019: 49 8b 4d 40 mov rcx,QWORD PTR [r13+0x40]
201d: 48 8b 41 08 mov rax,QWORD PTR [rcx+0x8]
2021: ff d0 call rax
2023: 48 83 c4 20 add rsp,0x20
2027: 5d pop rbp
2028: 41 5c pop r12
202a: 41 5d pop r13
202c: c3 ret
202d: 0f 1f 00 nop DWORD PTR [rax]

0000000000002030 <_start>:
2030: 48 83 ec 08 sub rsp,0x8
2034: 51 push rcx
2035: 52 push rdx
2036: 48 8d 3d c3 df ff ff lea rdi,[rip+0xffffffffffffdfc3]
# 0 <_text-0x2000>
203d: 48 8d 35 bc 2f 00 00 lea rsi,[rip+0x2fbc] # 5000
<_DYNAMIC>
2044: 59 pop rcx
2045: 5a pop rdx
2046: 51 push rcx
2047: 52 push rdx
2048: e8 13 00 00 00 call 2060 <_relocate>
204d: 5f pop rdi
204e: 5e pop rsi
204f: e8 ac ff ff ff call 2000 <_text>
2054: 48 83 c4 08 add rsp,0x8

0000000000002058 <.exit>:
2058: c3 ret
2059: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]

0000000000002060 <_relocate>:
2060: f3 0f 1e fa endbr64
2064: 48 8b 06 mov rax,QWORD PTR [rsi]
2067: 48 85 c0 test rax,rax
206a: 74 7e je 20ea <_relocate+0x8a>
206c: 48 83 c6 08 add rsi,0x8
2070: 31 d2 xor edx,edx
2072: 45 31 c0 xor r8d,r8d
2075: 31 c9 xor ecx,ecx
2077: eb 1a jmp 2093 <_relocate+0x33>
2079: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]
2080: 48 83 f8 07 cmp rax,0x7
2084: 74 72 je 20f8 <_relocate+0x98>
2086: 48 8b 46 08 mov rax,QWORD PTR [rsi+0x8]
208a: 48 83 c6 10 add rsi,0x10
208e: 48 85 c0 test rax,rax
2091: 74 1c je 20af <_relocate+0x4f>
2093: 48 83 f8 08 cmp rax,0x8
2097: 74 57 je 20f0 <_relocate+0x90>
2099: 48 83 f8 09 cmp rax,0x9
209d: 75 e1 jne 2080 <_relocate+0x20>
209f: 4c 8b 06 mov r8,QWORD PTR [rsi]
20a2: 48 8b 46 08 mov rax,QWORD PTR [rsi+0x8]
20a6: 48 83 c6 10 add rsi,0x10
20aa: 48 85 c0 test rax,rax
20ad: 75 e4 jne 2093 <_relocate+0x33>
20af: 48 89 d0 mov rax,rdx
20b2: 4c 09 c0 or rax,r8
20b5: 74 33 je 20ea <_relocate+0x8a>
20b7: 48 85 d2 test rdx,rdx
20ba: 74 44 je 2100 <_relocate+0xa0>
20bc: 4d 85 c0 test r8,r8
20bf: 74 3f je 2100 <_relocate+0xa0>
20c1: 48 85 c9 test rcx,rcx
20c4: 7e 24 jle 20ea <_relocate+0x8a>
20c6: 66 2e 0f 1f 84 00 00 cs nop WORD PTR [rax+rax*1+0x0]
20cd: 00 00 00
20d0: 83 7a 08 08 cmp DWORD PTR [rdx+0x8],0x8
20d4: 75 09 jne 20df <_relocate+0x7f>
20d6: 48 8b 02 mov rax,QWORD PTR [rdx]
20d9: 48 01 f8 add rax,rdi
20dc: 48 01 38 add QWORD PTR [rax],rdi
20df: 4c 29 c1 sub rcx,r8
20e2: 4c 01 c2 add rdx,r8
20e5: 48 85 c9 test rcx,rcx
20e8: 7f e6 jg 20d0 <_relocate+0x70>
20ea: 31 c0 xor eax,eax
20ec: c3 ret
20ed: 0f 1f 00 nop DWORD PTR [rax]
20f0: 48 8b 0e mov rcx,QWORD PTR [rsi]
20f3: eb 91 jmp 2086 <_relocate+0x26>
20f5: 0f 1f 00 nop DWORD PTR [rax]
20f8: 48 8b 16 mov rdx,QWORD PTR [rsi]
20fb: 48 01 fa add rdx,rdi
20fe: eb 86 jmp 2086 <_relocate+0x26>
2100: 48 b8 01 00 00 00 00 movabs rax,0x8000000000000001
2107: 00 00 80
210a: c3 ret
210b: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]


Note that the code at the _start entry point for some reason calls
efi_main with the arguments in different registers.

Can we (or do we want to) remove the other code (which comes from
libraries at link time)? Maybe but I am aware of some caveats. We can
discuss them in subsequent posts, as needed. There's also more that you
might need to know in order to generate the PE file. But, again, that
needs another post.
Post by James Harris
In that, the OutputString field holds the address of the UEFI function
THAT'S the question: how to find these function addresses ?
Post by James Harris
which can be called to write a string of 16-bit chars to the screen.
It is passed
    ECX = the STO table for the console (i.e. conout)
    EDX = the string to print
One thing UEFI gives the booted code is a pointer to the System Table.
The table is documented in section 4. It starts with the following
fields. (I included only as far as I needed.)
how. where and when will UEFI give me that?
Does the disassembly answer your remaining questions?
Post by James Harris
struc efi_systab ;UEFI system table
   .hdr:      resb efi_hdr_size
   .fvend:    resq 1 ;FirmwareVendor
   .frev:     resd 1 ;FirmwareRevision
              resd 1 ;padding
   .coninh:   resq 1 ;ConsoleInHandle
   .conin:    resq 1 ;ConIn
   .conouth:  resq 1 ;ConsoleOutHandle
   .conout:   resq 1 ;ConOut
endstruc
The header is
struc efi_hdr ;Generic header
   .Signature:    resq 1
   .Revision:     resd 1
   .HeaderSize:   resd 1
   .CRC32:        resd 1
                  resd 1  ;reserved
endstruc
;can you please show me the above again in my notation
don't mind I already got your type cast style :)
See above. Hopefully I've answered that question now but please say if not.
Post by James Harris
A note for anyone not familiar with C: where the spec in section 4.3.1 says
   CHAR16                           *FirmwareVendor;
   UINT32                           FirmwareRevision;
whereas FirmwareRevision in the second line is uint32, as you might
expect, the asterisk on the first line indicates that FirmwareVendor
is a /pointer/ to the stated type, char16, and so in this case the
field is 64-bit.
now I'm totally confused yet, CHAR16 means 64 bit ?
it seems it's just another 64 bit pointer.
No, in C terms FirmwareVendor is a pointer - i.e. an address - and what
it points to is of type char16.
Post by James Harris
Going back to the code, the System Table's conout field is a pointer
to a 'simple text output' (STO) table for the console, i.e. the screen
at boot time. An STO table includes the following fields.
struc efi_stop ;UEFI simple text output protocol
   .Reset:                   resq 1
   .OutputString:            resq 1
   .TestString:              resq 1
   .QueryMode:               resq 1
   .SetMode:                 resq 1
   .SetAttribute:            resq 1
   .ClearScreen:             resq 1
   .SetCursorPosition:       resq 1
   .EnableCursor:            resq 1
   .Mode:                    resq 1
endstruc
In that, the OutputString field holds the address of the UEFI function
which can be called to write a string of 16-bit chars to the screen.
It is passed
   ECX = the STO table for the console (i.e. conout)
   EDX = the string to print
Lo and behold, text appears! At least it does if one has everything
right. I can say more about that if anyone wants but for now I'll stop
here and not make this post any longer.
Feel free to add comments, questions and corrections.
thanks a lot, I now learned how to interpret your RESb/d/q :)
Those are not my terms but Nasm's. I didn't know you weren't familiar
with them but hopefully it's all now clear.
--
James Harris
wolfgang kern
2023-11-25 16:45:44 UTC
Permalink
On 25/11/2023 14:18, James Harris wrote:
..
Post by James Harris
where to ???  address of this call ms abi is
[rcx + efi_stop.OutputString] ???
No, take a look at the spec's section 2.3.4.1. Handoff State. It shows
that when UEFI calls an application it passes in the following.
Rcx - EFI_HANDLE
Rdx - EFI_SYSTEM_TABLE*
RSP - <return address>
I missed the very start of the whole story :)
when UEFI finds a valid file then it loads it and passes parameters:
In 32bit variant it comes on stack: [ESP+8] and [ESP].
Fine so far, with 64 bit mode: RCX and RDX.
But where would a Return go to ?
...
Post by James Harris
may the call return any useful parameter on the stack or in REGs ?
or can we just ignore all of it.
Yes, have a read through section 2.3.4.2. Detailed Calling Conventions.
It includes "Return values that fix [which I guess is supposed to be
"fit"] into 64-bits are returned in the Rax register" etc.
https://www.nasm.us/xdoc/2.16.01/html/nasmdoc5.html#section-5.9.1
...I'll show the final executable's full disassembly below.
I copied it away for a later compare with my thought solution.

...
Post by James Harris
Note that the code at the _start entry point for some reason calls
efi_main with the arguments in different registers.
Can we (or do we want to) remove the other code (which comes from
libraries at link time)? Maybe but I am aware of some caveats. We can
discuss them in subsequent posts, as needed. There's also more that you
might need to know in order to generate the PE file. But, again, that
needs another post.
Yes, I'm not enough familiar with NASM so it may take me a few hours to
learn that too, or just get a clue of what all must be in the UEFI.bin.

UEFI comes with C-libraries ?
I'd delete them immediate :)
Post by James Harris
Does the disassembly answer your remaining questions?
it may make me raise more questions later :)
Post by James Harris
Post by James Harris
struc efi_systab ;UEFI system table
   .hdr:      resb efi_hdr_size
   .fvend:    resq 1 ;FirmwareVendor
   .frev:     resd 1 ;FirmwareRevision
              resd 1 ;padding
   .coninh:   resq 1 ;ConsoleInHandle
   .conin:    resq 1 ;ConIn
   .conouth:  resq 1 ;ConsoleOutHandle
   .conout:   resq 1 ;ConOut
endstruc
The header is
struc efi_hdr ;Generic header
   .Signature:    resq 1
   .Revision:     resd 1
   .HeaderSize:   resd 1
   .CRC32:        resd 1
                  resd 1  ;reserved
endstruc
I try to convert the above: (pointed to by RDX)
efi_systab:
000 dq ptr to efi_header
008 dq STR16ptr to FirmwareVendor ;UTF16
010 d FirmwareRevision
014 d 0000_0000
018 dq handle for ConsoleIN
020 dq ptr tp ConsoleIN
028 dq handle for ConsoleOut
030 dq ptr to ConsoleOUT
----------------------------
efi_header:
000 dq Signature
008 dq Revision
010 q Headersize (in bytes) wouldn't it just say 0000001c ?
014 q CRC32 ;of what ???
018 q reserved ....
Post by James Harris
Post by James Harris
Going back to the code, the System Table's conout field is a pointer
to a 'simple text output' (STO) table for the console, i.e. the
screen at boot time. An STO table includes the following fields.
if this is pointed to by the ConsoleOUT aka conout above why has it a
different label yet ?
Post by James Harris
Post by James Harris
struc efi_stop ;UEFI simple text output protocol
   .Reset:                   resq 1
   .OutputString:            resq 1
   .TestString:              resq 1
   .QueryMode:               resq 1
   .SetMode:                 resq 1
   .SetAttribute:            resq 1
   .ClearScreen:             resq 1
   .SetCursorPosition:       resq 1
   .EnableCursor:            resq 1
   .Mode:                    resq 1
endstruc
In that, the OutputString field holds the address of the UEFI
function which can be called to write a string of 16-bit chars to the
screen. It is passed
   ECX = the STO table for the console (i.e. conout)
   EDX = the string to print
so the above 10 pointers contain usable function-addresses ?

and to get the address of OutputString I just need to perform
mov rax,[[RDX+0028]+0010] (in KESYS ptr[ptr] syntax)
IOW:
MOV rax,[RDX+0028]
MOV rax,[RAX+0010]
followed by:
MOV EDX, mystring
call RAX

me think I'm now nearer to the matter and the docs became more
understandable for me yet.
__
wolfgang
James Harris
2023-11-27 15:31:02 UTC
Permalink
Post by wolfgang kern
..
Post by James Harris
where to ???  address of this call ms abi is
[rcx + efi_stop.OutputString] ???
No, take a look at the spec's section 2.3.4.1. Handoff State. It shows
that when UEFI calls an application it passes in the following.
Rcx - EFI_HANDLE
Rdx - EFI_SYSTEM_TABLE*
RSP - <return address>
I missed the very start of the whole story :)
In 32bit variant it comes on stack: [ESP+8] and [ESP].
Fine so far, with 64 bit mode: RCX and RDX.
But where would a Return go to ?
AFAICT UEFI firmware is expected to have a 'boot manager' which knows
how to invoke one bootloader Application image after another until it
finds one which is willing to take control of the machine. Each
bootloader Application has to choose between two options:

Option 1: Go back to UEFI firmware. Two ways to do this: Either call
EFI_BOOT_SERVICES.Exit() (see sect 7.4.5) or return from efi_main.

Option 2: Take over the machine by calling ExitBootServices() and then,
most likely, pass control to an OS.

If we go back to the firmware then the boot manager could try a
different bootloader or drop to a CLI shell or present the user with a
TUI or GUI, etc. I guess that's configured on each machine.

For boot manager see section 3.

Status codes such as we might receive or we might pass back to EFI are
documented in appendix D.
Post by wolfgang kern
...
Post by James Harris
may the call return any useful parameter on the stack or in REGs ?
or can we just ignore all of it.
Yes, have a read through section 2.3.4.2. Detailed Calling
Conventions. It includes "Return values that fix [which I guess is
supposed to be "fit"] into 64-bits are returned in the Rax register" etc.
https://www.nasm.us/xdoc/2.16.01/html/nasmdoc5.html#section-5.9.1
...I'll show the final executable's full disassembly below.
I copied it away for a later compare with my thought solution.
...
Post by James Harris
Note that the code at the _start entry point for some reason calls
efi_main with the arguments in different registers.
Can we (or do we want to) remove the other code (which comes from
libraries at link time)? Maybe but I am aware of some caveats. We can
discuss them in subsequent posts, as needed. There's also more that
you might need to know in order to generate the PE file. But, again,
that needs another post.
Yes, I'm not enough familiar with NASM so it may take me a few hours to
learn that too, or just get a clue of what all must be in the UEFI.bin.
UEFI comes with C-libraries ?
I'd delete them immediate :)
I agree. The reloc library code is not wanted.

That said, AISI I don't need it at present because my own code is
position-independent but if I ever need to have non-PIC code then some
form of PE-reloc code would be necessary.
Post by wolfgang kern
Post by James Harris
Does the disassembly answer your remaining questions?
it may make me raise more questions later :)
:-)
Post by wolfgang kern
Post by James Harris
Post by James Harris
struc efi_systab ;UEFI system table
   .hdr:      resb efi_hdr_size
   .fvend:    resq 1 ;FirmwareVendor
   .frev:     resd 1 ;FirmwareRevision
              resd 1 ;padding
   .coninh:   resq 1 ;ConsoleInHandle
   .conin:    resq 1 ;ConIn
   .conouth:  resq 1 ;ConsoleOutHandle
   .conout:   resq 1 ;ConOut
endstruc
The header is
struc efi_hdr ;Generic header
   .Signature:    resq 1
   .Revision:     resd 1
   .HeaderSize:   resd 1
   .CRC32:        resd 1
                  resd 1  ;reserved
endstruc
I try to convert the above: (pointed to by RDX)
000 dq ptr to efi_header
008 dq STR16ptr to FirmwareVendor  ;UTF16
010 d  FirmwareRevision
014 d  0000_0000
018 dq handle for ConsoleIN
020 dq ptr tp ConsoleIN
028 dq handle for ConsoleOut
030 dq ptr to ConsoleOUT
----------------------------
000 dq Signature
008 dq Revision
010 q  Headersize (in bytes) wouldn't it just say 0000001c ?
014 q  CRC32                 ;of what ???
018 q reserved ....
If the 3-digit numbers are offsets then not quite. If the header is 24
bytes then when struc efi_systab specifies

.hdr: resb efi_hdr_size

it is reserving 24 bytes at the start of the system table.
Post by wolfgang kern
Post by James Harris
Post by James Harris
Going back to the code, the System Table's conout field is a pointer
to a 'simple text output' (STO) table for the console, i.e. the
screen at boot time. An STO table includes the following fields.
if this is pointed to by the ConsoleOUT aka conout above why has it a
different label yet ?
Not sure I understand the question but there's a common pattern in
calling UEFI firmware: that's to get an entry point from a structure and
then when calling the entry point pass it the address of the structure
that you got the entry point from. IOW, for structure S one would call

S->outputtext(S, "text")
Post by wolfgang kern
Post by James Harris
Post by James Harris
struc efi_stop ;UEFI simple text output protocol
   .Reset:                   resq 1
   .OutputString:            resq 1
   .TestString:              resq 1
   .QueryMode:               resq 1
   .SetMode:                 resq 1
   .SetAttribute:            resq 1
   .ClearScreen:             resq 1
   .SetCursorPosition:       resq 1
   .EnableCursor:            resq 1
   .Mode:                    resq 1
endstruc
In that, the OutputString field holds the address of the UEFI
function which can be called to write a string of 16-bit chars to
the screen. It is passed
   ECX = the STO table for the console (i.e. conout)
   EDX = the string to print
so the above 10 pointers contain usable function-addresses ?
I think the idea is that such structures can contain a mixture of
function addresses and data. For example, the .Mode field is a pointer
to data rather than a pointer to code.

Check the spec section 12.4.1 for details of the simple text output
structure.
Post by wolfgang kern
and to get the address of OutputString I just need to perform
mov rax,[[RDX+0028]+0010]  (in KESYS ptr[ptr] syntax)
  MOV rax,[RDX+0028]
  MOV rax,[RAX+0010]
  MOV EDX, mystring
  call RAX
The first arg should be in RCX and it needs to be the address of the
conout table - something like:

;Set up parms for text output
lea rdx, [rel msg_hello]
mov rcx, [r13 + efi_systab.conout]

;Make the call
mov rax, [rcx + efi_sto.OutputString]
call rax

One thing to add: until you dismiss the EFI firmware you should follow
the ABI calling conventions exactly including saving any callee-save
registers etc or your code might work on some machines but not others.

Feel free to correct me if I've got something wrong or mention anything
you find out. This is a learning experience for me, too!
--
James Harris
wolfgang kern
2023-11-27 22:43:26 UTC
Permalink
Post by James Harris
For boot manager see section 3.
OK.
Post by James Harris
Status codes such as we might receive or we might pass back to EFI are
documented in appendix D.
OK.
...
Post by James Harris
Post by wolfgang kern
Yes, I'm not enough familiar with NASM so it may take me a few hours
to learn that too, or just get a clue of what all must be in the
UEFI.bin.
UEFI comes with C-libraries ?
I'd delete them immediate :)
I agree. The reloc library code is not wanted.
That said, AISI I don't need it at present because my own code is
position-independent but if I ever need to have non-PIC code then some
form of PE-reloc code would be necessary.
My code was/is always position independent w/o self-relocation needs
because I know what goes where by using my brain instead of guessing
what assembler/compiler might do.
So I use immediate constants rather than dynamic RIP addressing.
Post by James Harris
Post by wolfgang kern
Post by James Harris
Does the disassembly answer your remaining questions?
it may make me raise more questions later :)
:-)
Questions may come soon :)

=========================================
Post by James Harris
Post by wolfgang kern
Post by James Harris
Post by James Harris
struc efi_systab ;UEFI system table
   .hdr:      resb efi_hdr_size
   .fvend:    resq 1 ;FirmwareVendor
   .frev:     resd 1 ;FirmwareRevision
              resd 1 ;padding
   .coninh:   resq 1 ;ConsoleInHandle
   .conin:    resq 1 ;ConIn
   .conouth:  resq 1 ;ConsoleOutHandle
   .conout:   resq 1 ;ConOut
endstruc
The header is
struc efi_hdr ;Generic header
   .Signature:    resq 1
   .Revision:     resd 1
   .HeaderSize:   resd 1
   .CRC32:        resd 1
                  resd 1  ;reserved
endstruc
I try to convert the above: (pointed to by RDX)
000 dq ptr to efi_header
008 dq STR16ptr to FirmwareVendor  ;UTF16
010 d  FirmwareRevision
014 d  0000_0000
018 dq handle for ConsoleIN
020 dq ptr tp ConsoleIN
028 dq handle for ConsoleOut
030 dq ptr to ConsoleOUT
----------------------------
000 dq Signature
008 dq Revision
010 q  Headersize (in bytes) wouldn't it just say 0000001c ?
014 q  CRC32                 ;of what ???
018 q reserved ....
=======================
Post by James Harris
If the 3-digit numbers are offsets then not quite. If the header is 24
bytes then when struc efi_systab specifies
  .hdr:      resb efi_hdr_size
it is reserving 24 bytes at the start of the system table.
I don't get that at all.
would you mind to correct my above translation in my format ?
...
Post by James Harris
Feel free to correct me if I've got something wrong or mention anything
you find out. This is a learning experience for me, too!
sing along: two blind mice... :)
coz my eyes became worse I need to zoom everything to 200%.
__
wolfgang
James Harris
2023-11-28 09:00:45 UTC
Permalink
...
Post by wolfgang kern
Post by James Harris
Post by wolfgang kern
UEFI comes with C-libraries ?
I'd delete them immediate :)
I agree. The reloc library code is not wanted.
That said, AISI I don't need it at present because my own code is
position-independent but if I ever need to have non-PIC code then some
form of PE-reloc code would be necessary.
My code was/is always position independent w/o self-relocation needs
because I know what goes where by using my brain instead of guessing
what assembler/compiler might do.
So I use immediate constants rather than dynamic RIP addressing.
You'll still need to refer to your own data (such as a Hello message)
and will need the address of its first character. How would you get the
right address if you don't work it out relative to RIP? AIUI the UEFI
firmware makes no guarantees as to where it loads you code.

...
Post by wolfgang kern
Post by James Harris
Post by wolfgang kern
Post by James Harris
struc efi_systab ;UEFI system table
   .hdr:      resb efi_hdr_size
   .fvend:    resq 1 ;FirmwareVendor
   .frev:     resd 1 ;FirmwareRevision
              resd 1 ;padding
   .coninh:   resq 1 ;ConsoleInHandle
   .conin:    resq 1 ;ConIn
   .conouth:  resq 1 ;ConsoleOutHandle
   .conout:   resq 1 ;ConOut
endstruc
The header is
struc efi_hdr ;Generic header
   .Signature:    resq 1
   .Revision:     resd 1
   .HeaderSize:   resd 1
   .CRC32:        resd 1
                  resd 1  ;reserved
endstruc
I try to convert the above: (pointed to by RDX)
000 dq ptr to efi_header
008 dq STR16ptr to FirmwareVendor  ;UTF16
010 d  FirmwareRevision
014 d  0000_0000
018 dq handle for ConsoleIN
020 dq ptr tp ConsoleIN
028 dq handle for ConsoleOut
030 dq ptr to ConsoleOUT
----------------------------
000 dq Signature
008 dq Revision
010 q  Headersize (in bytes) wouldn't it just say 0000001c ?
014 q  CRC32                 ;of what ???
018 q reserved ....
=======================
Post by James Harris
If the 3-digit numbers are offsets then not quite. If the header is 24
bytes then when struc efi_systab specifies
   .hdr:      resb efi_hdr_size
it is reserving 24 bytes at the start of the system table.
I don't get that at all.
 would you mind to correct my above translation in my format ?
I'm not sure I understand enough of your format to do that but if the
first three digits are the offsets in decimal then I think the System
Table in 64-bit mode would begin something like

000 24-byte Hdr
024 8-byte FirmwareVendor
032 4-byte FirmwareRevision
036 4-byte (unnamed padding)
040 8-byte ConsoleInHandle
048 8-byte ConIn
056 8-byte ConsoleOutHandle
064 8-byte ConOut
072 8-byte StandardErrorHandle
etc

That's based on the header at 4.2 and the System Table at 4.3 and
noticing that I added padding to align the next field and finding that
according to 2.3.1 the EFI_HEADER type is void* which is a pointer which
would be in C terms a pointer to void (which one could think of as a
pointer to an undeclared type but all we need to know is that it's a
pointer, i.e. 8-bytes in 64-bit mode). Phew!

*IOW* there's a lot of assumptions and a bunch of manual transcription
in the offsets I've written so they may not all be correct but hopefully
the idea and the comments provide enough info for you to get a little
further forward. Feel free to tell me of any errors and do check out the
types in 2.3.1 remembering that in C declarations an asterisk means a
pointer so T* means a pointer to T.

You may prefer shorter names, as I did. I took the above field names
from the spec's description of the System Table (at 4.3).
--
James Harris
wolfgang kern
2023-11-28 11:26:17 UTC
Permalink
Post by James Harris
...
Post by wolfgang kern
Post by James Harris
Post by wolfgang kern
UEFI comes with C-libraries ?
I'd delete them immediate :)
I agree. The reloc library code is not wanted.
That said, AISI I don't need it at present because my own code is
position-independent but if I ever need to have non-PIC code then
some form of PE-reloc code would be necessary.
My code was/is always position independent w/o self-relocation needs
because I know what goes where by using my brain instead of guessing
what assembler/compiler might do.
So I use immediate constants rather than dynamic RIP addressing.
You'll still need to refer to your own data (such as a Hello message)
and will need the address of its first character. How would you get the
right address if you don't work it out relative to RIP? AIUI the UEFI
firmware makes no guarantees as to where it loads you code.
while with 32bit it can use CS override.
with 64bit I need a one time "where I am" like:
CALL next
next:
POP R15
Post by James Harris
Post by wolfgang kern
Post by James Harris
Post by wolfgang kern
Post by James Harris
struc efi_systab ;UEFI system table
   .hdr:      resb efi_hdr_size
   .fvend:    resq 1 ;FirmwareVendor
   .frev:     resd 1 ;FirmwareRevision
              resd 1 ;padding
   .coninh:   resq 1 ;ConsoleInHandle
   .conin:    resq 1 ;ConIn
   .conouth:  resq 1 ;ConsoleOutHandle
   .conout:   resq 1 ;ConOut
endstruc
The header is
struc efi_hdr ;Generic header
   .Signature:    resq 1
   .Revision:     resd 1
   .HeaderSize:   resd 1
   .CRC32:        resd 1
                  resd 1  ;reserved
endstruc
I try to convert the above: (pointed to by RDX)
000 dq ptr to efi_header
008 dq STR16ptr to FirmwareVendor  ;UTF16
010 d  FirmwareRevision
014 d  0000_0000
018 dq handle for ConsoleIN
020 dq ptr tp ConsoleIN
028 dq handle for ConsoleOut
030 dq ptr to ConsoleOUT
----------------------------
000 dq Signature
008 dq Revision
010 q  Headersize (in bytes) wouldn't it just say 0000001c ?
014 q  CRC32                 ;of what ???
018 q reserved ....
=======================
Post by James Harris
If the 3-digit numbers are offsets then not quite. If the header is
24 bytes then when struc efi_systab specifies
   .hdr:      resb efi_hdr_size
it is reserving 24 bytes at the start of the system table.
I don't get that at all.
  would you mind to correct my above translation in my format ?
I'm not sure I understand enough of your format to do that but if the
first three digits are the offsets in decimal then I think the System
Table in 64-bit mode would begin something like
THANKS, let my try again (I always use HEX unless otherwise noted)
my size cast for 1/2/4/8/16/32/64 bytes is b/w/q/dq/qq/dqq/qqq.
[I use Q for quad since 58 years, it was used on 4 gate TTLchips]

rfi_systab:
dec|hex|size|
00 000 dq Signature
08 008 dq Revision
16 010 q Headersize ;0000_001c yet
20 014 q CRC32 ;CHKSUM of header?
;...
24 018 dq 8 byte FirmwareVendor at:[systab+[systab+16]]
32  020 q 4-byte FirmwareRevision at:[systab+[systab+16]]+008
36  024 q 4-byte (unnamed padding)
40 028 dq 8-byte ConsoleInHandle at:[systab+[systab+16]]+010
48  030 dq 8-byte ConIn +018
56  038 dq 8-byte ConsoleOutHandle +020
64  040 dq 8-byte ConOut +028
72  048 dq 8-byte StandardErrorHandle +030
etc
Post by James Harris
That's based on the header at 4.2 and the System Table at 4.3 and
noticing that I added padding to align the next field and finding that
according to 2.3.1 the EFI_HEADER type is void* which is a pointer which
would be in C terms a pointer to void (which one could think of as a
pointer to an undeclared type but all we need to know is that it's a
pointer, i.e. 8-bytes in 64-bit mode). Phew!
*IOW* there's a lot of assumptions and a bunch of manual transcription
in the offsets I've written so they may not all be correct but hopefully
the idea and the comments provide enough info for you to get a little
further forward. Feel free to tell me of any errors and do check out the
types in 2.3.1 remembering that in C declarations an asterisk means a
pointer so T* means a pointer to T.
You may prefer shorter names, as I did. I took the above field names
from the spec's description of the System Table (at 4.3).
OK, names are good for docs. I never use any literals in my code,
a hexadecimal address is enough name for me.
thanks, I'm now one step closer to start an overall conversion.
let us hope that i live long enough to complete that book :)
__
wolfgang
James Harris
2023-11-29 08:49:47 UTC
Permalink
...
Post by wolfgang kern
Post by James Harris
Post by wolfgang kern
Post by James Harris
I agree. The reloc library code is not wanted.
That said, AISI I don't need it at present because my own code is
position-independent but if I ever need to have non-PIC code then
some form of PE-reloc code would be necessary.
My code was/is always position independent w/o self-relocation needs
because I know what goes where by using my brain instead of guessing
what assembler/compiler might do.
So I use immediate constants rather than dynamic RIP addressing.
You'll still need to refer to your own data (such as a Hello message)
and will need the address of its first character. How would you get
the right address if you don't work it out relative to RIP? AIUI the
UEFI firmware makes no guarantees as to where it loads you code.
while with 32bit it can use CS override.
CALL next
POP R15
A few points:

1. x86 32-bit UEFI boot may not be widely supported. You may be better
to be able to boot a 64-bit UEFI OS loader and switch to 32-bit mode if
you want to start a 32-bit OS.

2. If UEFI load addresses aren't guaranteed then consider these scenarios:

a) Code at 1000, data at 1400
b) Code at 2000, data at 2400

What you need is data /relative to code/. Having a CS override wouldn't
help if you don't know where your code and data will be relative to CS.

3) If writing a 32-bit UEFI OS loader you could use call/pop to get data
address relative to wherever the code happens to have been loaded.

4) In x64 you don't need call/pop but can use RIP-relative references as in

lea rdx, [rel msg_hello]

for which the hex is

48 8d 15 e7 1f 00 00 lea rdx,[rip+0x1fe7]

5) The choices are to write PIC code or use relocation but bear in mind
that UEFI firmware has to see your final PE file as something it is
willing to load and execute so it might expect some flags or indicators
to tell it that the file is PIC or, failing that, that it comes with a
relocation section. UEFI is a lot more fussy than BIOS as to what it
will hand control to.
--
James Harris
wolfgang kern
2023-11-30 06:35:39 UTC
Permalink
...
Post by James Harris
Post by wolfgang kern
How would you get the right address if you don't work it out relative to RIP? AIUI the UEFI firmware makes no guarantees as to where it loads you code.
while with 32bit it can use CS override.
CALL next
POP R15
1. x86 32-bit UEFI boot may not be widely supported. You may be better
to be able to boot a 64-bit UEFI OS loader and switch to 32-bit mode if
you want to start a 32-bit OS.
a) Code at 1000, data at 1400
b) Code at 2000, data at 2400
I cannot believe that it may randomly load to different locations.
there must be some rule ie: dependent on media sector size.
Post by James Harris
What you need is data /relative to code/. Having a CS override wouldn't
help if you don't know where your code and data will be relative to CS.
when I write the file I know the data offset from within the code.
yours is 0x0400 for the text test. Mine will be much larger (>64K).
Post by James Harris
3) If writing a 32-bit UEFI OS loader you could use call/pop to get data
address relative to wherever the code happens to have been loaded.
4) In x64 you don't need call/pop but can use RIP-relative references as in
  lea rdx, [rel msg_hello]
for which the hex is
  48 8d 15 e7 1f 00 00     lea    rdx,[rip+0x1fe7]
OK I see it, but it is limited to 64k distance (same as in 32bit).

my one time self relocation used on start of several temporary dynamic
loaded modules (all static stuff is at fixed offsets as part of OS).

E8 0000_0000 CALL next
5A POP ESI
... adjust with known offsets

How would a 64bit version with larger than 64k code size look like?
Post by James Harris
5) The choices are to write PIC code or use relocation but bear in mind
that UEFI firmware has to see your final PE file as something it is
willing to load and execute so it might expect some flags or indicators
to tell it that the file is PIC or, failing that, that it comes with a
relocation section.
you mean UEFI will disassemble my code before loading it ?
we need to learn what exactly is required to make UEFI load and execute.
Post by James Harris
UEFI is a lot more fussy than BIOS as to what it will hand control to.
Oh yeah, that was the reason for I stopped all business with KESYS.

BTW, you have a working text test?
can you read out a few registers and print the values?
it would be interesting what we get on start, especially RSP.
__
wolfgang
James Harris
2023-11-30 10:30:17 UTC
Permalink
Post by wolfgang kern
...
Post by James Harris
Post by wolfgang kern
How would you get the right address if you don't work it out
relative to RIP? AIUI the UEFI firmware makes no guarantees as to
where it loads you code.
while with 32bit it can use CS override.
CALL next
POP R15
1. x86 32-bit UEFI boot may not be widely supported. You may be better
to be able to boot a 64-bit UEFI OS loader and switch to 32-bit mode
if you want to start a 32-bit OS.
a) Code at 1000, data at 1400
b) Code at 2000, data at 2400
I cannot believe that it may randomly load to different locations.
Why not? A UEFI app (such as a bootloader) is just an app as far as UEFI
is concerned. Where it gets loaded is not specified as it was with BIOS.
There's no simple 0x7c00 any more! The actual location of one's
bootloader will likely depend on who wrote the firmware, what /version/
of the firmware is being used, what drivers have been loaded beforehand,
what happened on the machine before it loaded our bootloader, etc.
Post by wolfgang kern
there must be some rule ie: dependent on media sector size.
Post by James Harris
What you need is data /relative to code/. Having a CS override
wouldn't help if you don't know where your code and data will be
relative to CS.
when I write the file I know the data offset from within the code.
yours is 0x0400 for the text test. Mine will be much larger (>64K).
Post by James Harris
3) If writing a 32-bit UEFI OS loader you could use call/pop to get
data address relative to wherever the code happens to have been loaded.
4) In x64 you don't need call/pop but can use RIP-relative references as in
   lea rdx, [rel msg_hello]
for which the hex is
   48 8d 15 e7 1f 00 00     lea    rdx,[rip+0x1fe7]
OK I see it, but it is limited to 64k distance (same as in 32bit).
Are you sure it's not +/- 2G? (Those zeroes...!)
Post by wolfgang kern
my one time self relocation used on start of several temporary dynamic
loaded modules (all static stuff is at fixed offsets as part of OS).
E8 0000_0000  CALL next
5A            POP  ESI
...           adjust with known offsets
That's fine, too, especially if used only once and not in tight nested
loops.
Post by wolfgang kern
How would a 64bit version with larger than 64k code size look like?
As I say, +/- 2G should be available.
Post by wolfgang kern
Post by James Harris
5) The choices are to write PIC code or use relocation but bear in
mind that UEFI firmware has to see your final PE file as something it
is willing to load and execute so it might expect some flags or
indicators to tell it that the file is PIC or, failing that, that it
comes with a relocation section.
you mean UEFI will disassemble my code before loading it ?
:-)

No, but remember that a bootloader has to be in PE format and such a
format has descriptive fields in its headers. The firmware will
naturally only load files if it likes what it sees in the headers.
Post by wolfgang kern
we need to learn what exactly is required to make UEFI load and execute.
That has to come from the spec because there are many implementations -
both present and future.
Post by wolfgang kern
Post by James Harris
UEFI is a lot more fussy than BIOS as to what it will hand control to.
Oh yeah, that was the reason for I stopped all business with KESYS.
BTW, you have a working text test?
can you read out a few registers and print the values?
it would be interesting what we get on start, especially RSP.
I have working text output but not number output in Asm. I could add it
but it doesn't seem useful for this as the contents of registers which
are not mentioned in the spec are unlikely to be consistent, AFIACS.
--
James Harris
wolfgang kern
2023-11-30 15:23:31 UTC
Permalink
On 30/11/2023 11:30, James Harris wrote:
...
Post by James Harris
Post by wolfgang kern
Post by James Harris
a) Code at 1000, data at 1400
b) Code at 2000, data at 2400
I cannot believe that it may randomly load to different locations.
Why not? A UEFI app (such as a bootloader) is just an app as far as UEFI
is concerned. Where it gets loaded is not specified as it was with BIOS.
Isn't the first thing it loads (or tries to) a file from the FAT32 root
directory with a certain name like "efi.bin" ?
Post by James Harris
There's no simple 0x7c00 any more! The actual location of one's
bootloader will likely depend on who wrote the firmware, what /version/
of the firmware is being used, what drivers have been loaded beforehand,
what happened on the machine before it loaded our bootloader, etc.
Yes, there should be something like HW detection, GDT/IDT, USB and VESA
stuff already mounted for the boot [ROM BIOS did that as well].

...
Post by James Harris
Post by wolfgang kern
Post by James Harris
   48 8d 15 e7 1f 00 00     lea    rdx,[rip+0x1fe7]
OK I see it, but it is limited to 64k distance (same as in 32bit).
Are you sure it's not +/- 2G? (Those zeroes...!)
yeah got me :) I checked on my disassembler and it tells this four bytes
are signed relative to following instruction (same as jump near/short),
Post by James Harris
Post by wolfgang kern
my one time self relocation used on start of several temporary dynamic
loaded modules (all static stuff is at fixed offsets as part of OS).
E8 0000_0000  CALL next
5E            POP  ESI
...           adjust with known offsets
That's fine, too, especially if used only once and not in tight nested
loops.
it should work almost equal in 64bit:
E8 0000_0000 CALL next ;sign-extended to 64 bit relative to L1
L1: 5E POP RSI ;get 64bit address of L1
Post by James Harris
Post by wolfgang kern
Post by James Harris
5) The choices are to write PIC code or use relocation but bear in
mind that UEFI firmware has to see your final PE file as something it
is willing to load and execute so it might expect some flags or
indicators to tell it that the file is PIC or, failing that, that it
comes with a relocation section.
I think relocation requirement is found in the header of an PE,
so if non is mentioned there UEFI should know how to act.
Post by James Harris
Post by wolfgang kern
you mean UEFI will disassemble my code before loading it ?
:-)
No, but remember that a bootloader has to be in PE format and such a
format has descriptive fields in its headers. The firmware will
naturally only load files if it likes what it sees in the headers.
you mean UEFI insists to have relocation ? what for ?

I have a copy from Herbert's shortest possible PE. I'll read it again.
Post by James Harris
Post by wolfgang kern
we need to learn what exactly is required to make UEFI load and execute.
That has to come from the spec because there are many implementations -
both present and future.
Post by wolfgang kern
Post by James Harris
UEFI is a lot more fussy than BIOS as to what it will hand control to.
Oh yeah, that was the reason for I stopped all business with KESYS.
BTW, you have a working text test?
can you read out a few registers and print the values?
it would be interesting what we get on start, especially RSP.
I have working text output but not number output in Asm. I could add it
but it doesn't seem useful for this as the contents of registers which
are not mentioned in the spec are unlikely to be consistent, AFIACS.
I remember back then when DOS once had similar issues, meanwhile start
conditions and reg-values are well documented and reliable (hugi comp).

I think UEFI will also either behave standardized or vanish soon.
what's missing on this matter is non C-styled detailed information.

OK so far I'll work on that after I assembled my new PC with parts
bought today [this may take a while].
__
wolfgang
Scott Lurndal
2023-11-30 15:41:09 UTC
Permalink
Post by wolfgang kern
...
Post by James Harris
Post by wolfgang kern
Post by James Harris
a) Code at 1000, data at 1400
b) Code at 2000, data at 2400
I cannot believe that it may randomly load to different locations.
Why not? A UEFI app (such as a bootloader) is just an app as far as UEFI
is concerned. Where it gets loaded is not specified as it was with BIOS.
Isn't the first thing it loads (or tries to) a file from the FAT32 root
directory with a certain name like "efi.bin" ?
Post by James Harris
There's no simple 0x7c00 any more! The actual location of one's
bootloader will likely depend on who wrote the firmware, what /version/
of the firmware is being used, what drivers have been loaded beforehand,
what happened on the machine before it loaded our bootloader, etc.
Yes, there should be something like HW detection, GDT/IDT, USB and VESA
stuff already mounted for the boot [ROM BIOS did that as well].
...
Post by James Harris
Post by wolfgang kern
Post by James Harris
   48 8d 15 e7 1f 00 00     lea    rdx,[rip+0x1fe7]
OK I see it, but it is limited to 64k distance (same as in 32bit).
Are you sure it's not +/- 2G? (Those zeroes...!)
yeah got me :) I checked on my disassembler and it tells this four bytes
are signed relative to following instruction (same as jump near/short),
Post by James Harris
Post by wolfgang kern
my one time self relocation used on start of several temporary dynamic
loaded modules (all static stuff is at fixed offsets as part of OS).
E8 0000_0000  CALL next
5E            POP  ESI
...           adjust with known offsets
That's fine, too, especially if used only once and not in tight nested
loops.
E8 0000_0000 CALL next ;sign-extended to 64 bit relative to L1
L1: 5E POP RSI ;get 64bit address of L1
Post by James Harris
Post by wolfgang kern
Post by James Harris
5) The choices are to write PIC code or use relocation but bear in
mind that UEFI firmware has to see your final PE file as something it
is willing to load and execute so it might expect some flags or
indicators to tell it that the file is PIC or, failing that, that it
comes with a relocation section.
I think relocation requirement is found in the header of an PE,
so if non is mentioned there UEFI should know how to act.
Post by James Harris
Post by wolfgang kern
you mean UEFI will disassemble my code before loading it ?
:-)
No, but remember that a bootloader has to be in PE format and such a
format has descriptive fields in its headers. The firmware will
naturally only load files if it likes what it sees in the headers.
you mean UEFI insists to have relocation ? what for ?
I have a copy from Herbert's shortest possible PE. I'll read it again.
Post by James Harris
Post by wolfgang kern
we need to learn what exactly is required to make UEFI load and execute.
That has to come from the spec because there are many implementations -
both present and future.
Post by wolfgang kern
Post by James Harris
UEFI is a lot more fussy than BIOS as to what it will hand control to.
Oh yeah, that was the reason for I stopped all business with KESYS.
BTW, you have a working text test?
can you read out a few registers and print the values?
it would be interesting what we get on start, especially RSP.
I have working text output but not number output in Asm. I could add it
but it doesn't seem useful for this as the contents of registers which
are not mentioned in the spec are unlikely to be consistent, AFIACS.
I remember back then when DOS once had similar issues, meanwhile start
conditions and reg-values are well documented and reliable (hugi comp).
I think UEFI will also either behave standardized or vanish soon.
what's missing on this matter is non C-styled detailed information.
OK so far I'll work on that after I assembled my new PC with parts
bought today [this may take a while].
Have you examined the UEFI source code? Most vendors base their implementation
on Tianocore.

https://github.com/tianocore

https://uefi.org/
Scott Lurndal
2023-11-30 17:06:35 UTC
Permalink
Post by wolfgang kern
I think UEFI will also either behave standardized or vanish soon.
what's missing on this matter is non C-styled detailed information.
UEFI -is already- standardized, and will never 'vanish'.

The alternative used by non-PC systems is generally uboot.

https://docs.u-boot.org/en/latest/
wolfgang kern
2023-12-01 06:25:08 UTC
Permalink
On 30/11/2023 16:41, Scott Lurndal wrote:
...
Post by Scott Lurndal
Post by wolfgang kern
I think UEFI will also either behave standardized or vanish soon.
what's missing on this matter is non C-styled detailed information.
Have you examined the UEFI source code?
it's in C so it's useless for bare metal machine code programmers.
Post by Scott Lurndal
Most vendors base their implementation
on Tianocore.
https://github.com/tianocore
thx, but I can't find anything useful for me there.
Post by Scott Lurndal
https://uefi.org/
I have this one pinned to my taskbar... and I started translation into
hex-dump/RBIL-styled info already.
__
wolfgang
James Harris
2023-12-01 13:24:03 UTC
Permalink
Post by wolfgang kern
...
Post by James Harris
Post by wolfgang kern
Post by James Harris
a) Code at 1000, data at 1400
b) Code at 2000, data at 2400
I cannot believe that it may randomly load to different locations.
Why not? A UEFI app (such as a bootloader) is just an app as far as
UEFI is concerned. Where it gets loaded is not specified as it was
with BIOS.
Isn't the first thing it loads (or tries to) a file from the FAT32 root
directory with a certain name like "efi.bin" ?
We were talking about /where/ UEFI loads the PE file. And you've kept
that prior discussion in. So the change to /what/ UEFI loads is unexpected.

But on the topic of what gets loaded ... all I can say is I don't know!
I presume each machine can be configured with a list of things to try.
Just like a BIOS can be configured to look for a floppy, an optical
drive, then the first hard disk, or similar, I believe a UEFI machine
can be configured to try a series of potentially bootable media and
files until it gets to one which works.

That said, there are some standard locations, most notably

/efi/boot/

Files in there called

bootARCH.efi

are meant to be bootable for the specified architecture. ARCH could be
such as ia32 or x64 (and yes, 3-char architecture names are OK).

I have been testing with qemu. No idea how to configure a boot list on
it but by default it will load either

/efi/boot/bootx64.efi

or

/startup.nsh

where the latter is a series of textual commands. startup.nsh is akin to
autoexec.bat.
Post by wolfgang kern
Post by James Harris
There's no simple 0x7c00 any more! The actual location of one's
bootloader will likely depend on who wrote the firmware, what
/version/ of the firmware is being used, what drivers have been loaded
beforehand, what happened on the machine before it loaded our
bootloader, etc.
Yes, there should be something like HW detection, GDT/IDT, USB and VESA
stuff already mounted for the boot [ROM BIOS did that as well].
...
Post by wolfgang kern
Post by James Harris
Post by wolfgang kern
Post by James Harris
5) The choices are to write PIC code or use relocation but bear in
mind that UEFI firmware has to see your final PE file as something
it is willing to load and execute so it might expect some flags or
indicators to tell it that the file is PIC or, failing that, that it
comes with a relocation section.
I think relocation requirement is found in the header of an PE,
so if non is mentioned there UEFI should know how to act.
Note, though, that in the previously posted disassembly the PE file
appeared to do its own relocation.

I think the services UEFI might use to load PE files are

EFI_BOOT_SERVICES.LoadImage
EFI_BOOT_SERVICES.StartImage

but from a quick scan the standards don't seem to say what either of
them does about relocation. Probably nothing, would be my guess.
Post by wolfgang kern
Post by James Harris
Post by wolfgang kern
you mean UEFI will disassemble my code before loading it ?
:-)
No, but remember that a bootloader has to be in PE format and such a
format has descriptive fields in its headers. The firmware will
naturally only load files if it likes what it sees in the headers.
you mean UEFI insists to have relocation ? what for ?
I don't know what characteristics the OS loader (or any UEFI
Application) is expected to have. I tried removing some of the library
code you now have in the disassembly I posted before. I managed to get
rid of some of it but there came a point where the firmware would no
longer load the PE file.

It's something I've to get back to. All I can say just now is that the
PE file which the firmware wouldn't load had no reloc section.
Post by wolfgang kern
I have a copy from Herbert's shortest possible PE. I'll read it again.
Sounds good. Hope you'll feed back what you come up with.
--
James Harris
wolfgang kern
2023-12-01 14:33:03 UTC
Permalink
Post by James Harris
Post by wolfgang kern
Isn't the first thing it loads (or tries to) a file from the FAT32
root directory with a certain name like "efi.bin" ?
We were talking about /where/ UEFI loads the PE file. And you've kept
that prior discussion in.
sorry, I forgot to skip that out
Post by James Harris
So the change to /what/ UEFI loads is unexpected.
But on the topic of what gets loaded ... all I can say is I don't know!
I presume each machine can be configured with a list of things to try.
Just like a BIOS can be configured to look for a floppy, an optical
drive, then the first hard disk, or similar, I believe a UEFI machine
can be configured to try a series of potentially bootable media and
files until it gets to one which works.
so I can assume PCs with UEFI_only_ have F11 boot-options as well.
Post by James Harris
That said, there are some standard locations, most notably
  /efi/boot/
is this folder tree mandatory or just convenient ?
Post by James Harris
Files in there called
  bootARCH.efi
are meant to be bootable for the specified architecture. ARCH could be
such as ia32 or x64 (and yes, 3-char architecture names are OK).
OK.
Post by James Harris
I have been testing with qemu. No idea how to configure a boot list on
it but by default it will load either
  /efi/boot/bootx64.efi
or
  /startup.nsh
where the latter is a series of textual commands. startup.nsh is akin to
autoexec.bat.
that's interesting, I'll check what this one can do.
...

[about relocation needs]
Post by James Harris
Note, though, that in the previously posted disassembly the PE file
appeared to do its own relocation.
you mean the RIP here? so it has no relocation table?
Post by James Harris
I think the services UEFI might use to load PE files are
EFI_BOOT_SERVICES.LoadImage
EFI_BOOT_SERVICES.StartImage
but from a quick scan the standards don't seem to say what either of
them does about relocation. Probably nothing, would be my guess.
Ok, after several month I may have enough info converted to try boot on
real hardware.
...
Post by James Harris
It's something I've to get back to. All I can say just now is that the
PE file which the firmware wouldn't load had no reloc section.
I'm confused yet :)
Post by James Harris
Post by wolfgang kern
I have a copy from Herbert's shortest possible PE. I'll read it again.
Sounds good. Hope you'll feed back what you come up with.
Sure, but as said above it will take some time before I can run tests.

I go back now to do conversion yet, there might come more questions...

thanks for helping me old blind crossing a heavy traffic road
__
wolfgang
James Harris
2023-12-02 12:11:35 UTC
Permalink
...
Post by wolfgang kern
Post by James Harris
That said, there are some standard locations, most notably
   /efi/boot/
is this folder tree mandatory or just convenient ?
According to

https://wiki.osdev.org/UEFI

Bootable UEFI applications

The boot order NVRAM variables determine where firmware will look for
UEFI applications to be launched at boot. Although this can be changed
(for example, an OS installer might customize the boot entry for the
hard drive to which it was installed) firmware typically looks for a
UEFI application named "BOOTIA32.efi" (for 32-bit applications) or
"BOOTX64.efi" (for 64-bit applications) stored in the "/EFI/BOOT" path
in the boot device's file system. This is the default path and name for
OVMF.

...
Post by wolfgang kern
[about relocation needs]
Post by James Harris
Note, though, that in the previously posted disassembly the PE file
appeared to do its own relocation.
you mean the RIP here?  so it has no relocation table?
No, remember the very start of the disassembly (at label _start) began with

0000000000002030 <_start>:
2030: 48 83 ec 08 sub rsp,0x8
2034: 51 push rcx
2035: 52 push rdx
2036: 48 8d 3d c3 df ff ff lea
rdi,[rip+0xffffffffffffdfc3] # 0 <_text-0x2000>
203d: 48 8d 35 bc 2f 00 00 lea rsi,[rip+0x2fbc] #
5000 <_DYNAMIC>
2044: 59 pop rcx
2045: 5a pop rdx
2046: 51 push rcx
2047: 52 push rdx
2048: e8 13 00 00 00 call 2060 <_relocate>

which sure sounds like it's doing its own relocation rather than UEFI
firmware doing the relocation.

I haven't followed it through but it looks as though it is using the
dynamic relocation section (_DYNAMIC) from the PE file.
--
James Harris
wolfgang kern
2023-12-03 10:06:37 UTC
Permalink
Post by James Harris
Post by wolfgang kern
Post by James Harris
That said, there are some standard locations, most notably
   /efi/boot/
is this folder tree mandatory or just convenient ?
According to
  https://wiki.osdev.org/UEFI
Bootable UEFI applications
The boot order NVRAM variables determine where firmware will look for
UEFI applications to be launched at boot. Although this can be changed
(for example, an OS installer might customize the boot entry for the
hard drive to which it was installed) firmware typically looks for a
UEFI application named "BOOTIA32.efi" (for 32-bit applications) or
"BOOTX64.efi" (for 64-bit applications) stored in the "/EFI/BOOT" path
in the boot device's file system. This is the default path and name for
OVMF.
OK, it may be mandatory. But you know me, I'm a revolutioneer and always
try to overrule "reported" needs and get rid of all useless stuff.
I let you know how it works on real hardware when I'm there.
Post by James Harris
Post by wolfgang kern
[about relocation needs]
Post by James Harris
Note, though, that in the previously posted disassembly the PE file
appeared to do its own relocation.
you mean the RIP here?  so it has no relocation table?
No, remember the very start of the disassembly (at label _start) began with
    2030:    48 83 ec 08              sub    rsp,0x8
    2034:    51                       push   rcx
    2035:    52                       push   rdx
    2036:    48 8d 3d c3 df ff ff     lea
rdi,[rip+0xffffffffffffdfc3]    # 0 <_text-0x2000>
    203d:    48 8d 35 bc 2f 00 00     lea    rsi,[rip+0x2fbc]        #
5000 <_DYNAMIC>
    2044:    59                       pop    rcx
    2045:    5a                       pop    rdx
    2046:    51                       push   rcx
    2047:    52                       push   rdx
    2048:    e8 13 00 00 00           call   2060 <_relocate>
which sure sounds like it's doing its own relocation rather than UEFI
firmware doing the relocation.
I haven't followed it through but it looks as though it is using the
dynamic relocation section (_DYNAMIC) from the PE file.
so your PE contains a relocation table, how large is it ?
how would a relocation table look like if there is not a single
reference in it ?

I checked Herbert's PE, seems it's a 32 bit format w/o any relocation.
__
wolfgang

James Harris
2023-12-02 12:15:43 UTC
Permalink
On 29/11/2023 08:49, James Harris wrote:

...
Post by James Harris
1. x86 32-bit UEFI boot may not be widely supported. You may be better
to be able to boot a 64-bit UEFI OS loader and switch to 32-bit mode if
you want to start a 32-bit OS.
I knew I'd seen somewhere that 32-bit UEFI boot may not be widely
supported. I think it was at

https://forum.osdev.org/viewtopic.php?f=1&t=56668
--
James Harris
wolfgang kern
2023-12-03 09:51:32 UTC
Permalink
Post by James Harris
Post by James Harris
1. x86 32-bit UEFI boot may not be widely supported. You may be better
to be able to boot a 64-bit UEFI OS loader and switch to 32-bit mode
if you want to start a 32-bit OS.
Yes, that's what I have to try first even my 32-bit KESYS can switch to
LM and back, the whole thing will need a complete rewrite for the
opposite direction, I may this do later, (if I live long enough)
Post by James Harris
I knew I'd seen somewhere that 32-bit UEFI boot may not be widely
supported. I think it was at
  https://forum.osdev.org/viewtopic.php?f=1&t=56668
I see, perhaps just contemporary on mobiles and earlier.
__
wolfgang
Loading...