Chris wrote:
here comes the patch for the first wave of Use class size reduction.
I have split it into 3 files, corresponding to
- header changes
- implementation changes
- applications
nice!
This at the moment does not contain the description how the
size of the Use class will be reduced from 16 to 12 bytes,
I am going to send that in a separate patch.
Right, sounds great.
This wave primarily consists of changes that are needed
to allocate objects with a variable number of embedded s.
Ok.
Where the number of s is constant in the lifetime of an object,
I preferred to keep the ‘new Instr(…)’ syntax. Otherwise I have
introduced static ‘Create(…)’ methods, which are used instead of the
‘new Instr(…)’ construct.
The bad thing about this is that it is inconsistent. I’d prefer to
have consistent use of ::Create for all IR classes to make it easier
to learn the API. That said, a temporary moment of inconsistency is
ok: you could commit this part, then switch the remaining classes over.
Yeah, this is a wart, but it is only temporary as you say.
I already have some sed scripts that aid the transition 
These replace the constructors and the
constructors become private/protected. The arguments of the ‘Create’
methods are identical to the corresponding constructors.
Essentially at the moment all introduced 'operator new’s end up
calling
‘::operator new(size_t)’, so there should be no functionality change
at all.
This will change in subsequent waves.
This basically looks good, some thoughts for the future:
- Please (eventually) don’t make the ‘operator new’ override be
public. I’d actually prefer all memory alloc/dealloc stuff to be done
privately to the (vmcore) .cpp files, not exposed through the header.
Okay. I guess these relpacement ‘operator new’ will go away in favor of
Foo::Create methods.
- Eventually we’ll need to make the dtor private as well.
Okay. So no more 'delete xxx;". But then we must be careful to
never call the Foo::Destroy() method on a NULL ptr.
- Make sure that make check and some reasonable subset of llvm-test
passes with this patch
I have never run llvm-test in the past. Is it just checking it out and
following a readme?
Finally, please update llvm-gcc 4.2 as well when you commit this.
I’ll update clang after you commit.
Well, I have patches for both (attached). But I am caught between a rock and a
hard place, because Xcode 2.4.1-built llvm-gcc ICEs-out on the
Release llvm (which is needed as per README-LLVM). So I have a hard
time to run those tests. I keep trying. Possibly softlinking the
Release dirs to the Debug ones.
gcc.diff (9.69 KB)
clang.diff (12.4 KB)
...
> 3) Make sure that make check and some reasonable subset of llvm-test
> passes with this patch
I have never run llvm-test in the past. Is it just checking it out and
following a readme?
After building an llvm-gcc (4.2.1), see below, I tried running this. I
configured:
./configure --with-llvmsrc=$LLVM_TOP/llvm --with-llvmobj=$LLVM_TOP/
llvm --srcdir=$LLVM_TOP/test-suite --with-llvmgccdir=$LLVM_TOP/install
--with-externals=$LLVM_TOP/externals
Then 'make', it complained that some .bc file could not be made.
Then I put $LLVM_TOP/install and $LLVM_TOP/llvm/Release/bin im my
path, but now
checking for C++ compiler default output file name... configure:
error: C++ compiler cannot create executables
invoking g++ directly gives:
ggreif$ g++ jj.cpp
/usr/bin/ld: can't locate file for: -lstdc++
I am stuck. What can I do to run the tests?
> Finally, please update llvm-gcc 4.2 as well when you commit this.
> I'll update clang after you commit.
Well, I have patches for both (attached). But I am caught between a
rock and a
hard place, because Xcode 2.4.1-built llvm-gcc ICEs-out on the
Release llvm (which is needed as per README-LLVM). So I have a hard
time to run those tests. I keep trying. Possibly softlinking the
Release dirs to the Debug ones.
This approach works. I get past bootstrapping llvm-gcc.
Thanks and cheers,
Gabor
heisenbug wrote:
...
3) Make sure that make check and some reasonable subset of llvm-test
passes with this patch
I have never run llvm-test in the past. Is it just checking it out and
following a readme?
After building an llvm-gcc (4.2.1), see below, I tried running this. I
configured:
./configure --with-llvmsrc=$LLVM_TOP/llvm --with-llvmobj=$LLVM_TOP/
llvm --srcdir=$LLVM_TOP/test-suite --with-llvmgccdir=$LLVM_TOP/install
--with-externals=$LLVM_TOP/externals
Then 'make', it complained that some .bc file could not be made.
Then I put $LLVM_TOP/install and $LLVM_TOP/llvm/Release/bin im my
path, but now
checking for C++ compiler default output file name... configure:
error: C++ compiler cannot create executables
invoking g++ directly gives:
ggreif$ g++ jj.cpp
/usr/bin/ld: can't locate file for: -lstdc++
I am stuck. What can I do to run the tests?
Did you do a make install? It is probably searching for libstdc++ from
the g++ you just built.
I usually add a "--program-prefix=llvm-" to configure, and then I can
just do a make install into /usr/local without
interfering with system's gcc/g++.
But for running llvm-test I think you should have system's gcc/g++ on
your path and not llvm-gcc/llvm-g++.
If I understand correctly, the purpose of the testsuite is to test
system gcc/g++ vs. LLC vs. CBE vs. JIT.
Best regards,
--Edwin
heisenbug wrote:
> ...
>>> 3) Make sure that make check and some reasonable subset of llvm-test
>>> passes with this patch
>> I have never run llvm-test in the past. Is it just checking it out and
>> following a readme?
> After building an llvm-gcc (4.2.1), see below, I tried running this. I
> configured:
> ./configure --with-llvmsrc=$LLVM_TOP/llvm --with-llvmobj=$LLVM_TOP/
> llvm --srcdir=$LLVM_TOP/test-suite --with-llvmgccdir=$LLVM_TOP/install
> --with-externals=$LLVM_TOP/externals
> Then 'make', it complained that some .bc file could not be made.
> Then I put $LLVM_TOP/install and $LLVM_TOP/llvm/Release/bin im my
> path, but now
> checking for C++ compiler default output file name... configure:
> error: C++ compiler cannot create executables
> invoking g++ directly gives:
> ggreif$ g++ jj.cpp
> /usr/bin/ld: can't locate file for: -lstdc++
> I am stuck. What can I do to run the tests?
Did you do a make install? It is probably searching for libstdc++ from
the g++ you just built.
yes. into $LLVM_TOP/install
I usually add a "--program-prefix=llvm-" to configure, and then I can
just do a make install into /usr/local without
interfering with system's gcc/g++.
Okay, will try this route
But for running llvm-test I think you should have system's gcc/g++ on
your path and not llvm-gcc/llvm-g++.
If I understand correctly, the purpose of the testsuite is to test
system gcc/g++ vs. LLC vs. CBE vs. JIT.
point taken. thanks!
have consistent use of ::Create for all IR classes to make it easier
to learn the API. That said, a temporary moment of inconsistency is
ok: you could commit this part, then switch the remaining classes over.
Yeah, this is a wart, but it is only temporary as you say.
I already have some sed scripts that aid the transition
No problem.
1) Please (eventually) don't make the 'operator new' override be
public. I'd actually prefer all memory alloc/dealloc stuff to be done
privately to the (vmcore) .cpp files, not exposed through the header.
Okay. I guess these relpacement 'operator new' will go away in favor of
Foo::Create methods.
yep.
2) Eventually we'll need to make the dtor private as well.
Okay. So no more 'delete xxx;". But then we must be careful to
never call the Foo::Destroy() method on a NULL ptr.
I'd suggest making foo::Destroy() be non-virtual. In the implementation it can just check for 'this == null' and return immediately if so. Doing this also helps with the "eliminate the vtable pointer" subproject to shrink "Value".
3) Make sure that make check and some reasonable subset of llvm-test
passes with this patch
I have never run llvm-test in the past. Is it just checking it out and
following a readme?
It's pretty easy, take a look at docs/TestingGuide.html
Finally, please update llvm-gcc 4.2 as well when you commit this.
I'll update clang after you commit.
Well, I have patches for both (attached). But I am caught between a rock and a hard place, because Xcode 2.4.1-built llvm-gcc ICEs-out on the Release llvm (which is needed as per README-LLVM). So I have a hard time to run those tests. I keep trying. Possibly softlinking the Release dirs to the Debug ones.
This is strange.
When you commit, please email llvmdev/commits with an email that says
"API CHANGE" in the subject line with info on how to upgrade out-of-
tree projects (e.g. vmkit).
Okay, will include the tcsh scripts I have written.
Nice!
-Chris
> heisenbug wrote:
> > ...
> >>> 3) Make sure that make check and some reasonable subset of llvm-test
> >>> passes with this patch
> >> I have never run llvm-test in the past. Is it just checking it out and
> >> following a readme?
> > After building an llvm-gcc (4.2.1), see below, I tried running this. I
> > configured:
> > ./configure --with-llvmsrc=$LLVM_TOP/llvm --with-llvmobj=$LLVM_TOP/
> > llvm --srcdir=$LLVM_TOP/test-suite --with-llvmgccdir=$LLVM_TOP/install
> > --with-externals=$LLVM_TOP/externals
> > Then 'make', it complained that some .bc file could not be made.
> > Then I put $LLVM_TOP/install and $LLVM_TOP/llvm/Release/bin im my
> > path, but now
> > checking for C++ compiler default output file name... configure:
> > error: C++ compiler cannot create executables
> > invoking g++ directly gives:
> > ggreif$ g++ jj.cpp
> > /usr/bin/ld: can't locate file for: -lstdc++
> > I am stuck. What can I do to run the tests?
> Did you do a make install? It is probably searching for libstdc++ from
> the g++ you just built.
yes. into $LLVM_TOP/install
> I usually add a "--program-prefix=llvm-" to configure, and then I can
> just do a make install into /usr/local without
> interfering with system's gcc/g++.
Okay, will try this route
> But for running llvm-test I think you should have system's gcc/g++ on
> your path and not llvm-gcc/llvm-g++.
> If I understand correctly, the purpose of the testsuite is to test
> system gcc/g++ vs. LLC vs. CBE vs. JIT.
point taken. thanks!
Whatever I try I get something like this:
ggreif$ cd MultiSource/
ggreif$ make
make[2]: *** No rule to make target `Output/be.bc', needed by `Output/
burg.linked.rbc'. Stop.
make[1]: *** [Burg/.makeall] Error 2
make: *** [Applications/.makeall] Error 2
What can this be?
This is the wonderful indicator that the llvm configure script didn't find llvm-gcc. Make sure that llvm-gcc is in your path, and the rerun the llvm configure script. llvm-test should magically start working.
-Chris
Hmmm, yes, it did. BUT...
it is still a complete chaos.
Here is the nightly report:
Program | GCCAS Bytecode LLC compile LLC-BETA
compile JIT codegen | GCC CBE LLC LLC-BETA JIT | GCC/
CBE GCC/LLC GCC/LLC-BETA LLC/LLC-BETA
SignlessTypes/cast | 0.0690 * *
* * | 0.03 * * *
* | n/a n/a n/a n/a
SignlessTypes/cast-bug | 0.0160 * *
* * | 0.00 * * *
* | n/a n/a n/a n/a
SignlessTypes/cast2 | 0.0124 * *
* * | 0.00 * * *
* | n/a n/a n/a n/a
SignlessTypes/ccc | 0.0148 * *
* * | 0.00 * * *
* | n/a n/a n/a n/a
SignlessTypes/div | 0.0298 * *
* * | 0.00 * * *
* | n/a n/a n/a n/a
SignlessTypes/factor | 0.0354 * *
* * | 0.00 * * *
* | n/a n/a n/a n/a
SignlessTypes/rem | 0.2015 * *
* * | 0.00 * * *
* | n/a n/a n/a n/a
SignlessTypes/shr | 0.0278 * *
* * | 0.00 * * *
* | n/a n/a n/a n/a
Vector/SSE/sse.expandfft | 0.1480 8592 0.1545
* * | 0.60 0.85 0.52 *
0.68 | 0.71 1.15 n/a n/a
Vector/SSE/sse.isamax | 0.0586 3500 0.0553
* * | 0.00 0.00 0.00 *
0.08 | - - n/a n/a
Vector/SSE/sse.shift | 0.0145 980 0.0170
* * | 0.00 * 0.00 *
0.04 | n/a - n/a n/a
Vector/SSE/sse.stepfft | 0.1606 9776 0.1771
* * | 0.84 0.98 0.65 *
0.85 | 0.86 1.29 n/a n/a
Vector/build | 0.0169 * *
* * | 0.00 * * *
* | n/a n/a n/a n/a
Vector/build2 | 0.0423 * *
* * | 1.35 * * *
* | n/a n/a n/a n/a
Vector/divides | 0.0204 * *
* * | 0.00 * * *
* | n/a n/a n/a n/a
Vector/multiplies | 0.0505 * *
* * | 9.40 * * *
* | n/a n/a n/a n/a
Vector/simple | 0.0331 * *
* * | 0.00 * * *
* | n/a n/a n/a n/a
Vector/sumarray | 0.0228 * *
* * | 0.00 * * *
* | n/a n/a n/a n/a
Vector/sumarray-dbl | 0.0241 * *
* * | 0.00 * * *
* | n/a n/a n/a n/a
A typical failure is logged thus:
/Users/ggreif/llvm/Debug/bin/llvm-ld -L/lib/gcc/i686-apple-
darwin8.11.1/ -L/lib Output/build.linked.bc -lc -lcrtend -o
Output/build.llvm
llvm-ld: error: Cannot find library 'crtend'
make[3]: [Output/build.llvm.bc] Error 1 (ignored)
wc: Output/build.llvm.bc: open: No such file or directory
/Users/ggreif/llvm/Debug/bin/llc -f Output/build.llvm.bc -o Output/
build.llc.s
/Users/ggreif/llvm/Debug/bin/llc: bitcode didn't read correctly.
Reason: could not open file
make[3]: [Output/build.llc.s] Error 1 (ignored)
gcc Output/build.llc.s -o Output/build.llc -lm -mdynamic-no-pic -
fomit-frame-pointer
i686-apple-darwin8-gcc-4.0.1: Output/build.llc.s: No such file or
directory
make[3]: [Output/build.llc] Error 1 (ignored)
/Users/ggreif/test-suite/RunSafely.sh 500 0 /dev/null Output/build.out-
llc Output/build.llc
/Users/ggreif/test-suite/DiffOutput.sh "/Users/ggreif/llvm/Debug/bin/
fpcmp " llc build
******************** TEST (llc) 'build' FAILED! ********************
Execution Context Diff:
/Users/ggreif/llvm/Debug/bin/fpcmp: FP Comparison failed, not a
numeric difference between '1' and '/'
******************** TEST (llc) 'build' ****************************
rm -f Output/build.exe-llc
cp Output/build.diff-llc Output/build.exe-llc
cp: Output/build.diff-llc: No such file or directory
make[3]: [Output/build.exe-llc] Error 1 (ignored)
head -n 100 Output/build.exe-llc >> Output/
build.nightly.llc.report.txt
head: Output/build.exe-llc: No such file or directory
make[3]: [Output/build.nightly.llc.report.txt] Error 1 (ignored)
/Users/ggreif/test-suite/RunSafely.sh 500 0 /dev/null Output/build.out-
jit /Users/ggreif/llvm/Debug/bin/lli -force-interpreter=false --
disable-core-files -time-passes -stats -info-output-file=/Users/ggreif/
test-suite/SingleSource/UnitTests/Vector/Output/build.out-jit.info
Output/build.llvm.bc
/Users/ggreif/test-suite/DiffOutput.sh "/Users/ggreif/llvm/Debug/bin/
fpcmp " jit build
******************** TEST (jit) 'build' FAILED! ********************
Execution Context Diff:
/Users/ggreif/llvm/Debug/bin/fpcmp: FP Comparison failed, not a
numeric difference between '1' and '/'
******************** TEST (jit) 'build' ****************************
rm -f Output/build.exe-jit
cp Output/build.diff-jit Output/build.exe-jit
cp: Output/build.diff-jit: No such file or directory
make[3]: [Output/build.exe-jit] Error 1 (ignored)
head -n 100 Output/build.exe-jit >> Output/
build.nightly.jit.report.txt
head: Output/build.exe-jit: No such file or directory
make[3]: [Output/build.nightly.jit.report.txt] Error 1 (ignored)
/Users/ggreif/llvm/Debug/bin/llc -march=c Output/build.llvm.bc -o
Output/build.cbe.c -f
/Users/ggreif/llvm/Debug/bin/llc: bitcode didn't read correctly.
Reason: could not open file
make[3]: [Output/build.cbe.c] Error 1 (ignored)
gcc Output/build.cbe.c -o Output/build.cbe -O3 -fno-strict-aliasing
-fno-inline -mdynamic-no-pic -fomit-frame-pointer -lcrtend
i686-apple-darwin8-gcc-4.0.1: Output/build.cbe.c: No such file or
directory
make[3]: [Output/build.cbe] Error 1 (ignored)
/Users/ggreif/test-suite/RunSafely.sh 500 0 /dev/null Output/build.out-
cbe Output/build.cbe
/Users/ggreif/test-suite/DiffOutput.sh "/Users/ggreif/llvm/Debug/bin/
fpcmp " cbe build
******************** TEST (cbe) 'build' FAILED! ********************
Execution Context Diff:
/Users/ggreif/llvm/Debug/bin/fpcmp: FP Comparison failed, not a
numeric difference between '1' and '/'
******************** TEST (cbe) 'build' ****************************
rm -f Output/build.exe-cbe
cp Output/build.diff-cbe Output/build.exe-cbe
cp: Output/build.diff-cbe: No such file or directory
make[3]: [Output/build.exe-cbe] Error 1 (ignored)
head -n 100 Output/build.exe-cbe >> Output/
build.nightly.cbe.report.txt
head: Output/build.exe-cbe: No such file or directory
make[3]: [Output/build.nightly.cbe.report.txt] Error 1 (ignored)
cat Output/build.nightly.compile.report.txt Output/
build.nightly.nat.report.txt Output/build.nightly.llc.report.txt
Output/build.nightly.jit.report.txt Output/
build.nightly.cbe.report.txt > Output/build.nightly.report.txt
As promised here comes the algorithmic part of the
project.
I have documented the way how the User object can be
recovered from an array of Use objects.
I have included a reference implementation in Haskell,
along with a randomized test suite, which passes.
This is just for those who want to manually
prove the correctness of the C++ algorithm.
If you wish I can remove (or move to another location)
the Haskell part.
A new file, Use.cpp contains the codec for the
bit-encoding. These routines will be called by the
memory allocation routines for User and by Use::getUser().
The interface is not finalized yet, I am thinking of
a User <--> Use friendship to make the new routines
private, and we definitely need some casting to get this
done. I hope to nicely hide the ugliness behind a friendly
facade.
When this part is done and checked in to trunk, I shall
begin to implement the proposed memory layout, but without
the memory reduction for now. Instead the two algorithms
will run in parallel, backing up each other and comparing
the data, asserting on discrepancies.
Here is the review link:
<http://llvm.org/viewvc/llvm-project?view=rev&revision=49380>
Cheers,
Gabor
Thinking about it, it is probably a poor decision to
put the tag bits into the Value*. This incurs unnecessary
overhead as the Use::Val member is accessed frequently.
My final proposal will say that the Prev member will be used
to carry the tag bits necessary for recovering the User*.
This member is far less frequently accessed and syntactically
better isolated.
Cheers,
Gabor
