[PATCH] patch to compile llvm-gcc using nightly tester script(NewNightlyTester.pl)

Rajika_Kumarasiri2 · July 6, 2008, 3:47pm

hello every body.

Here with I have attached the patch which compile the llvm-gcc using nightly tester script. This patch add the following capabilities to the NewNightlyTester.pl script.

Checkout the llvm-gcc4.2 source from the SVN.
Compile the checkout llvm-gcc4.2 source tree.
Gather the configure/make out put informations.
Add the (3) informations to the %hash_of_data hash to be posted to the server, of course for further processing.

Please review the patch and let me know your comments. There are few small TODOs (ex: get the LOC of llvm-gcc etc…,) to be added. I’ll update the patch with that soon.

Thank you

-Rajika

NewNightlyTest.patch (13.8 KB)

Rajika_Kumarasiri2 · July 7, 2008, 6:10pm

hello everybody,

I have added few improvements to my patch. Please review the new patch directly.
Thanks!

-Rajika

NewNightlyTest_v2.patch (13.7 KB)

void · July 7, 2008, 6:26pm

Hi Rajika,

A few stylistic comments. I'll let others comment on the algorithm for now:

     if ($VERBOSE) {
       print "( time -p $SVNCMD/llvm/trunk llvm; cd llvm/projects ; " .
- "$SVNCMD/test-suite/trunk llvm-test ) > $COLog 2>&1\n";
+ "$SVNCMD/test-suite/trunk llvm-test ; cd ../../;" .
+ "$SVNCMD/llvm-gcc-4.2/trunk dst-directory ) > $COLog 2>&1\n";
     }
     system "( time -p $SVNCMD/llvm/trunk llvm; cd llvm/projects ; " .
- "$SVNCMD/test-suite/trunk llvm-test ) > $COLog 2>&1\n";
+ "$SVNCMD/test-suite/trunk llvm-test ; cd ../../;" .
+ "$SVNCMD/llvm-gcc-4.2/trunk dst-directory ) > $COLog 2>&1\n";

This could be done with something like this:

     my $cmd = "( time -p $SVNCMD/llvm/trunk llvm; cd llvm/projects ; " .
              "$SVNCMD/llvm-gcc-4.2/trunk dst-directory; " .
              "cd llvm/projects; " .
              "$SVNCMD/test-suite/trunk llvm-test ) > $COLog 2>&1\n";

print $cmd if ($VERBOSE);
system $cmd;

This could be done in other spots that do similar things as well.
Also, why not name the LLVM-GCC source directory something other than
"dst-directory"? Maybe "llvm-gcc.src" or similar? You add a few
extraneous newlines in the code. Please remove those.

- my $CVSCMD = "$NICE cvs $CVSOPT -d $CVSRootDir co -P $CVSCOOPT";
+ my $CVSCMD = "$NICE cvs $CVSOPT -d $CVSRootDir co -P $CVSCOOPT";
# TODO, do we still maintain the CVS tree ?

80-column violation here and in other places. Put the comment before
here. (And, no, we don't maintain the CVS tree anymore.)

-bw

Rajika_Kumarasiri2 · July 7, 2008, 6:49pm

hi Bill,
Thanks for the comments. I’ll update the patch according to that.

-Rajika

void · July 7, 2008, 7:46pm

Thanks

One last thing. It seems like the patch has a lot of whitespace
changes. Could you create the diff with the -b option?

-bw

tonic · July 7, 2008, 9:55pm

Rajika,

A couple of comments:

- You should provide a way to specify where llvm-gcc is built (just like llvm).

- I would highly recommend allowing the user to only update llvm-gcc and not check it out from scratch each time. Checking out llvm-gcc is very time consuming. You would need to make sure that llvm and llvm-gcc have the same rev number and nuke the llvm obj/install dirs so you get a clean build. Sure svn issues can happen, but they are rare and your script changes should be able to catch those errors.

- Remove the cvs stuff, its dead. Feel free to clean up the script and remove ALL cvs stuff.

- I think if LLVM_GCC_CONFIGURE is set and the flag -nollvmgcc is not set,
you should throw an error.

- You should not try to add the configure options yourself, this is not worth it in my opinion (maybe in the future if you want to base it off the target triple or something) but for now.. keep it simple. You should only add the --enable-llvm since only the script knows where its built.

- What happens if LLVMGCCDIR is set? It will use that over the llvm-gcc you have built. In fact, I don't think you have reconfigured llvm at all after building llvm-gcc.

- Checkout to something more meaningful then dst-directory.

- We don't care about warnings, file sizes or loc of llvm-gcc. If you want
   to keep any stats you can keep track of config, build times, and build
   status. The rest is stuff we don't really want to track in the database
  (too much info!).

Also, how have you tested this?? Where is the patch to the DB schema and accept scripts?

-Tanya

Rajika_Kumarasiri2 · July 8, 2008, 4:36am

Thanks

One last thing. It seems like the patch has a lot of whitespace
changes. Could you create the diff with the -b option?

Sure, I’ll keep this in mind, thanks!

-Rajika

Rajika_Kumarasiri2 · July 8, 2008, 6:33pm

Hi Tanya,
See my comments in line,

Rajika,

A couple of comments:

You should provide a way to specify where llvm-gcc is built (just like
llvm).

I think this is given by LLVMGCCDIR . And also what about the location where the actual llvm-source is kept(which will be updated by the script) ?

I would highly recommend allowing the user to only update llvm-gcc and
not check it out from scratch each time. Checking out llvm-gcc is very
time consuming.

So where should we maintain the llvm-source tree in a localmachine, so that the script will update that copy? Are we going to get that location using an ENV varible ? And the installtion path will be given by LLVMGCCDIR, or to a default location if it is not set.

You would need to make sure that llvm and llvm-gcc have
the same rev number and nuke the llvm obj/install dirs so you get a clean
build. Sure svn issues can happen, but they are rare and your script
changes should be able to catch those errors.

I can keep track of the revision number of llvm and llvm-gcc once they different from each other I can throw an error, but why should they be synchronised to be the same revision number? I am still not clear how that(i.e. each of the revision number in llvm and llvm-gcc of course) affect the llvm-gcc build.

Remove the cvs stuff, its dead. Feel free to clean up the
script and remove ALL cvs stuff.

Sure I’ll do this and I’ll be renaming all the CVS stuffs into SVN ex: the SVN check out log is still CVS-Log.txt(it will be SVN-Log.txt etc…)

I think if LLVM_GCC_CONFIGURE is set and the flag -nollvmgcc is not set,
you should throw an error.

Well what if a user has a value for the environment varible LLVM_GCC_CONFIGURE but he doesn’t want to build the llvm-gcc tree ?
What I thought was if a user needs to build llvm-gcc he/she should set -nollvmgcc flag, if it going to happens in the normal way (i.e. they buid llvm, llvm-gcc, test etc…, all the time) we can simply drop this flag. I added this following -nobuild flag for llvm.
Or simply if a user set -nobuild flag we can simply build neither llvm nor llvm-gcc ?

You should not try to add the configure options yourself, this is not
worth it in my opinion (maybe in the future if you want to base it off the
target triple or something) but for now… keep it simple. You should only
add the --enable-llvm since only the script knows where its built.

Ok

What happens if LLVMGCCDIR is set?

So that , that LLVMGCCDIR will be used to install the llvm-gcc, discarding the normal default location sepecified by the user (see above)

It will use that over the llvm-gcc
you have built.

In fact, I don’t think you have reconfigured llvm at all
after building llvm-gcc.

You mean to run the Deja GNU tests right? . OK I’ll add that part as well

Checkout to something more meaningful then dst-directory.

What about llvm-gcc as Bill suggested ?

We don’t care about warnings, file sizes or loc of llvm-gcc. If you want
to keep any stats you can keep track of config, build times, and build
status. The rest is stuff we don’t really want to track in the database
(too much info!).

Ok, I’ll not worry about LOC of llvm-gcc etc…

Also, how have you tested this??

Yeah, I ran it (with -nosubmit) and had a look at the output.log which indicates the values of the newly added varibles for llvm-gcc.
And I have the following TEXT files under tests results directory.

2008-07-07-Build-Log.txt
2008-07-07-CVS-Log.txt
2008-07-07-Dejagnu-testrun.log
2008-07-07-Dejagnu-testrun.sum
2008-07-07-DejagnuTests-Log.txt
2008-07-07-LLVMGCC-Build-Log.txt
2008-07-07-LLVMGCC-Warnings.txt
2008-07-07-MultiSource-Performance.txt
2008-07-07-MultiSource-ProgramTest.txt
2008-07-07 23:11 2008-07-07-MultiSource-Tests.txt
2008-07-07 23:14 2008-07-07-Olden-tests.txt
2008-07-07 23:11 2008-07-07-Performance.txt
2008-07-07 23:04 2008-07-07-SingleSource-Performance.txt
2008-07-07-SingleSource-ProgramTest.txt
2008-07-07-SingleSource-Tests.txt
2008-07-07-Tests.txt
2008-07-07-Warnings.txt

In which LLVMGCC-* are the newly created result files for LLVM-GCC build.

Where is the patch to the DB schema and
accept scripts?

I didn’t look into the server side yet, will send that patch soon.

Thanks for the comments and help. I’ll attacch the updated patch soon.

-Rajika