[lld] -emit-yaml doesnot contain linker added symbols specified with command line options

Hi,

Right now, linker added symbols specified by the -u option do not endup in the output YAML file.

This is because the target specific Writers dont get called, which creates the undefined atoms.

I am in the process of adding more options and I would like the atoms created internally by the options available in the output YAML file.

The options that I am trying to consider for the linker internal atoms to appear in the output YAML file are :-

a) Replace Writer::addFiles to Reader::addFiles

This would achieve any file to be added to the list of input files that is processed.

                               (or)
b) Create a seperate API in the inputGraph for the driver to add lld::Files, that are consumed by the Universal driver and that gets
added to the list of File objects that is being considered

                               (or)

c) Handle all the command line switches of -u or any atoms that we want to see them appear in the YAML file as a seperate pass.

I dont like the option (c), because we are using a hammer to kill this problem.

Thanks

Shankar Easwaran

Shankar,

The LinkingContext has a addImplictFiles() method that is supposed to call the Writer and give it a chance to add any implicit files. Is the problem that the -u atoms are not attached to that implicit file? Or that the implicit file is not getting added? Or that this got lost in the transition from InputFiles to InputGraph?

-Nick

Hi Nick,

The problem is when the -emit-yaml option is used, the writer is set to the YAML writer.

The YAML writer doesnot have anything to add here.

The problem can be solved by having the YAML writer append a list of undefined atoms specified by the -u option, but the problem I have is each flavor has extra command line options
for which it wants to create a DefinedAtom/UndefinedAtom. The flavor also may want to add extra linker internal files in the future.

I prefer addImplicitFiles calling the reader to add more files, which means addFiles API moves to the reader.

Thanks

Shankar Easwaran

Hi Nick,

The problem is when the -emit-yaml option is used, the writer is set to the YAML writer.

Ah!

The YAML writer does not have anything to add here.

The problem can be solved by having the YAML writer append a list of undefined atoms specified by the -u option, but the problem I have is each flavor has extra command line options
for which it wants to create a DefinedAtom/UndefinedAtom. The flavor also may want to add extra linker internal files in the future.

I prefer addImplicitFiles calling the reader to add more files, which means addFiles API moves to the reader.

But won’t that fail too if you were using a YAML Reader and ELF Writer?

Since we are talking about files/atoms that are created because of command line options, perhaps the Driver should be creating the files/atoms. In the case of -u, it would be nice to group them all in one file named “command line option -u” so that any error messages about undefined symbols says the references was from “command line option -u” (as opposed to the -u UndefinedAtoms come from a generic internal file). Also for the case of -u, the Driver can make generic UndefinedAtoms. Other options may need platform specific atoms which may be created with the help of static Writer methods.

-Nick

Hi Nick,

The YAML writer does not have anything to add here.

The problem can be solved by having the YAML writer append a list of undefined atoms specified by the -u option, but the problem I have is each flavor has extra command line options
for which it wants to create a DefinedAtom/UndefinedAtom. The flavor also may want to add extra linker internal files in the future.

I prefer addImplicitFiles calling the reader to add more files, which means addFiles API moves to the reader.

But won’t that fail too if you were using a YAML Reader and ELF Writer?  

Yeah :frowning:

Since we are talking about files/atoms that are created because of command line options, perhaps the Driver should be creating the files/atoms. 

Currently the Driver has a notion of all the files that it processes are all Linker Inputs. InputGraph could have an additional function, to just add to it a list of files created implicitly by the linker. The UniversalDriver can then look at this list and append the list of files to the vector of files and then process the InputFiles.

 In the case of -u, it would be nice to group them all in one file named “command line option -u” so that any error messages about undefined symbols says the references was from “command line option -u”  (as opposed to the -u UndefinedAtoms come from a generic internal file). 

Agree.

 Also for the case of -u, the Driver can make generic UndefinedAtoms. 

ELF would want to treat the undefined symbols as a weak undefined symbols. So the driver cant create undefined atoms.

Below is an example :-

$cat 1.c
int _start() { return 0; }
$gcc -c 1.c
$ld -u myundef 1.o
==> Does not throw any error, the resolver was hinted that myundef was a undefined weak symbol.

 Other options may need platform specific atoms which may be created with the help of static Writer methods.

We wouldnt be able to override the functionality with each flavor, if we have a static method, or you had thought of something else ?

Thanks

Shankar Easwaran

Hi Nick,

 Also for the case of -u, the Driver can make generic UndefinedAtoms. 

ELF would want to treat the undefined symbols as a weak undefined symbols. So the driver cant create undefined atoms.

Below is an example :-

$cat 1.c
int _start() { return 0; }
$gcc -c 1.c
$ld -u myundef 1.o
==> Does not throw any error, the resolver was hinted that myundef was a undefined weak symbol.

Wow. Reading the gnu ld man page, that is not obvious. How can you make a non-weak undefined on the command line? That is, how can you force and error if something is not defined?

 Other options may need platform specific atoms which may be created with the help of static Writer methods.

We wouldnt be able to override the functionality with each flavor, if we have a static method, or you had thought of something else ?

I’m not sure when we would want polymorphism. If you wanted it in this case, there could be a method on LinkingContext to create a file of UndefinedAtoms, and ELFLinkingContext could override that method to make weak undefines.

-Nick

Hi Nick,

$cat 1.c
int _start() { return 0; }
$gcc -c 1.c
$ld -u myundef 1.o
==> Does not throw any error, the resolver was hinted that myundef was a undefined weak symbol.
Wow. Reading the gnu ld man page, that is not obvious. How can you make a non-weak undefined on the command line? That is, how can you force and error if something is not defined?

I think the linker will barf only if the input file contained an undefined symbol and not by any other means.

  Other options may need platform specific atoms which may be created with the help of static Writer methods.

We wouldnt be able to override the functionality with each flavor, if we have a static method, or you had thought of something else ?

I’m not sure when we would want polymorphism. If you wanted it in this case, there could be a method on LinkingContext to create a file of UndefinedAtoms, and ELFLinkingContext could override that method to make weak undefines.

Yes this will work for undefined atoms that need to be supported by all flavors. All the implicit files that are needed would be added to a vector of files in the inputGraph,
addImplicitFiles would splice/prepend to the list of input files which it already has.

Do you think that we still want the Writer::addFiles API, as this already covers the functionalities that is needed.

Thanks

Shankar Easwaran

At least with mach-o there is a small advantage to having the MachOWriter create some atoms (as opposed to the MachOLinkingContext). For instance, the Writer needs to know which atom is the entry point. Current the MachOWriter creates the CRuntimeFile which has an atom X with a reference to UndefinedAtom for “_main”. When the Writer is called to generate the output file, it can look at the internal atom X and see what the reference now points to and use that as the entry. If the CRuntimeFile was created by the LinkingContext there would have to be an interface for the Writer to extract the entry target atom.

-Nick