Windows clang assembler and sysv_abi

This is a bit special case, so it is probably not very exciting.

I have an existing Unix project that I am porting to Windows, this includes a handful of assembler versions. For example:


extern void function_assembler(P1, P2, P3, P4, P5, P6);

static void function_in_C( ... ) 
    function_assembler(P1, P2, P3, P4, P5, P6);


.intel_syntax noprefix
.global function_assembler

.p2align 6
        movups  xmm0, xmmword ptr [rdi] # Expected P1 in rdi, but is in rdx.

Because all the .s files are from Unix, I’ve had to convert the argument passing from

Unix: rdi, rsi, rdx, rcx, r8, r9

Windows: rdx, rcx, r8, r9, rsp+0x28, rsp+0x30

(and as well, saving rdi, rsi, xmm6-xmm15)

Then I stumbled across __attribute__((sysv_abi)).

I would be very neat if I could just tell the compiler to use the Unix parameter order: rdi, rsi, rdx, rcx, r8, r9

So I define the assembler function in C, as:

extern void __attribute__((sysv_abi)) function_assembler(P1, P2, P3, P4, P5, P6);

but I am disappointed to see when I dump registers entering function_assembler: P1 is still in rdx and not as I had hoped, rdi.

Is there some way to get this to work? It would be very beneficial to not have to tweak all the .s files for Windows. (or have a separate copy of .s files for Windows)

Compile time, I use clang to compile all the source files, .c and .s, into a libkern.a. Then at the end, link a driver.c with MSVC and the libkern.a for the final driver. Oh yeah, this is in kernel mode. But since both blake3.c and blake3.s are inside libkern.a, I would not have thought that should matter.

vs2019 uses clang 12, and vs2021 uses clang 15 - both passes P1 in rdx.

Should it work? Did I do something wrong? Or will it never work?


The approach you are trying does not work because the LLVM compiler transforms C into assembly, not assembly into assembly.

Even though you changed the function declaration with this special attribute, the implementation remains in an assembly file, not a C file, so the (C to assembly) compiler knows nothing about what is inside; the translation from assembly to machine code is the job of the system assembler (i.e. GNU assembler)

However there exists a project within LLVM to provide traditional assembling and disassembling : Intro to the LLVM MC Project - The LLVM Project Blog

I have never used it myself, but perhaps it could be possible to disassemble your (Windows) .s files into LLVM IR, and then use clang or llc to recompile to (Unix) assembly? Perhaps others who know more about it can weigh in…

Ah is that the case. I just assumed that the compiler is in control of calling a function, be it C or assembler, so if I say that my assembler function expects rdi, rsi, … it would shuffle things around for me, before calling it.
I don’t want to change the assembly file at all, but tell C how to call it. It sounded like __attribute__((sysv_abi)) did just that.

I’ve had moderate success with

long ret;
    __asm__ (
	"mov %[p6], %%r9  \n\t"
	"mov %[p5], %%r8  \n\t"
	"mov %[p4], %%rcx \n\t"
	"mov %[p3], %%rdx \n\t"
	"mov %[p2], %%rsi \n\t"
	"mov %[p1], %%rdi \n\t"
	"call zfs_blake3_compress_xof_sse2 \n\t"
	: "=a"(ret)
	: [p1]"m"(cv), [p2]"m"(block), [p3]"m"(block_len), [p4]"m"(counter), [p5]"m"(flags), [p6]"m"(out)
	: "%xmm6"

So I’ll continue down that path and make some MACROS.

This should work exactly as you intend it to - without a more detailed example, I don’t see why it wouldn’t.

Yes, the compiler can do this.

Wine used to rely on being able to generate code for ms_abi functions on non-Windows OSes (both for calling functions, and for implementing functions that receive arguments with this calling convention), while I’m not aware if use of sysv_abi on Windows is as common (so there could be bugs there).

I made a test example out of your code, see Compiler Explorer. This shows declaring two extern functions with both sysv_abi and ms_abi, and calling them, and building this code for both linux and windows. In the case of calling the native calling convention, it is just turned into a tail call, while for the foreign calling convention, it does reshuffle arguments for you, exactly as you had expected.

I didn’t proofread all the reshuffling code that is generated, but it does look like it is doing the right thing.


Thanks for the “compiler explorer” example, so I tried your example in the local build, and this popped up:

clang.exe -o roger.o -c roger.c
roger.c:2:21: warning: 'sysv_abi' calling convention is not supported for this target [-Wignored-attributes]
void __attribute__((sysv_abi)) sysv_abi_func(int p1, int p2, int p3, int p4, int p5, int p6, int p7);

I wonder if I’ve been ignoring that warning somewhere. Interestingly, you showed me what to do with
-target x86_64-windows-msvc
I am not sure what effect that would have if I just added it to my build? Whats the default if not specified?
Anyway, the output generated looks like:

      ed: 44 8b 8c 24 08 01 00 00       movl    264(%rsp), %r9d
      f5: 44 8b 84 24 00 01 00 00       movl    256(%rsp), %r8d
      fd: 8b 4c 24 1c                   movl    28(%rsp), %ecx
     101: 8b 54 24 18                   movl    24(%rsp), %edx
     105: 8b 74 24 14                   movl    20(%rsp), %esi
     109: 8b 7c 24 10                   movl    16(%rsp), %edi
     10d: 89 04 24                      movl    %eax, (%rsp)
     110: e8 00 00 00 00                callq   0x115 <call_sysv_abi+0xa5>

Then to make sure clang-cl.exe does the same, had to change compile target to something like:
clang-cl.exe -o roger.o -c roger.c --target=x86_64-pc-windows-msvc19.33.31629

But it also produced what looks like the correct output.

Cheers! That really helped. I will check if I can just stick --target=x86_64-pc-windows-msvc19.33.31629 on the project as whole, or, just for those .C files that needs it.

That’s odd - I wonder what target you’re building for, if you get such warnings.

To see the default, run e.g. clang --version. With the official LLVM releases for Windows, this example works without any warnings without specifying --target, with at least the 12.0, 13.0 and 15.0 releases. What build/distribution of Clang are you using?

clang-cl.exe --version
clang version 12.0.0
Target: i686-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\Llvm\bin

when in vs2019.

Currently the test case above does not work in my sources, but works separately. So I’ve been cutting things down to see what happens, so far, just:

+#include "blake3_impl.h"
void __attribute__((ms_abi)) ms_abi_func(int p1, int p2, int p3, int p4, int p5, int p6, int p7);
void __attribute__((sysv_abi)) sysv_abi_func(int p1, int p2, int p3, int p4, int p5, int p6, int p7);

Will stop it from working, ie, using the same include. Naturally, it’s not that header file itself, but presumably, all the includes it includes, which I guess will end up inside windows includes. Perhaps they define some default calling type that overrides whatever I pick later. Deeper I go…

OK this is all because past-lundman did a “smart” thing to work out some incompatibilities.

#ifdef _MSC_VER
#define	__attribute__(X)

So the feature works like a charm with less “smart” code. Thanks!

Right, that explains the warning about this convention not being supported, if the compiler defaults to i686.

Ok, great, that explains the rest of it!