Xilinx zynq-7000 (7030) as a Gallium3D LLVM FPGA target

i was just writing this:
http://www.gp32x.com/board/index.php?/topic/60228-replicating-the-success-of-the-openpandora-discussion-v20/

when something that just occurred to me, half way through, and i would
greatly appreciate some help evaluating whether it's feasible.

put these together:
http://www.xilinx.com/products/silicon-devices/epp/zynq-7000/index.htm
http://llvm.org/releases/2.7/docs/ReleaseNotes.html
http://en.wikipedia.org/wiki/Gallium3D#LLVM_usage
http://wiki.opengraphics.org

and it may be the case that completing the software for the
OpenGraphics Project is a matter of hitting "compile". not only that,
but a fully FSF Hardware-Endorseable Laptop with decent 3D Graphics
Engine would be as equally simple (the only FSF Hardware-Endorsed
laptop product is that Loongson Leemote, by virtue of it having a 2D
PCI Graphics IC and a 900mhz MIPS).

the logic goes as follows:
* one of the Gallium3D targets is LLVM.
* one of LLVM's targets is Xilinx FPGAs (MicroBlaze).
* the zynq-7000 7030 has 125k FPGA Logic Gates (and more)
* with a TFP410 as the DVI driver, the OGP is done!

now, given that this appears to be "too easy", i'd really _really_
appreciate some help checking the facts. and, also, if it turns out
to be feasible, assessing roughly what the performance might be.

also, most importantly: what am i missing? :slight_smile:

many many thanks,

l.

Luke Kenneth Casson Leighton wrote:

i was just writing this:
http://www.gp32x.com/board/index.php?/topic/60228-replicating-the-success-of-the-openpandora-discussion-v20/

when something that just occurred to me, half way through, and i would
greatly appreciate some help evaluating whether it's feasible.

put these together:
http://www.xilinx.com/products/silicon-devices/epp/zynq-7000/index.htm
http://llvm.org/releases/2.7/docs/ReleaseNotes.html
http://en.wikipedia.org/wiki/Gallium3D#LLVM_usage
http://wiki.opengraphics.org

and it may be the case that completing the software for the
OpenGraphics Project is a matter of hitting "compile". not only that,
but a fully FSF Hardware-Endorseable Laptop with decent 3D Graphics
Engine would be as equally simple (the only FSF Hardware-Endorsed
laptop product is that Loongson Leemote, by virtue of it having a 2D
PCI Graphics IC and a 900mhz MIPS).

the logic goes as follows:
* one of the Gallium3D targets is LLVM.
* one of LLVM's targets is Xilinx FPGAs (MicroBlaze).
* the zynq-7000 7030 has 125k FPGA Logic Gates (and more)
* with a TFP410 as the DVI driver, the OGP is done!

now, given that this appears to be "too easy", i'd really _really_
appreciate some help checking the facts. and, also, if it turns out
to be feasible, assessing roughly what the performance might be.

also, most importantly: what am i missing? :slight_smile:

The way in which Gallium3D targets LLVM, is that it waits until it receives the shader program from the application, then compiles that down to LLVM IR. That's too late to start synthesizing hardware (unless you're planning to ship an FPGA as the graphics card, in which case reprogramming is still too slow, and it'll be too expensive).

A more accurate approach would be to say, "Well, Mesa is a software implementation of the whole OpenGL stack, right? Why not compile it to verilog using something like c-to-verilog.com (also powered by LLVM) then synthesize that?" The answer is that the resulting code will be far too inefficient to make a useful modern graphics card, and even if it were, you'll have to do lots of work to make it communicate of PCI and DVI, etc. Hardware isn't POSIX after all.

Nick

nick.... the Zynq-7000 series of Dual-Core Cortex A9 800mhz 28nm CPUs
have an on-board Series 7 Artix-7 or Kinect-7 FPGA (depending on the
Zynq range). and that's on the same silicon IC :slight_smile: so, price is not
an object any more [assuming reasonable volume].

here's the eetimes article which mentions that the low-end version of
the 7000 series will be under $USD 15 in mass-volume:
http://www.eetimes.com/electronics-news/4213637/Xilinx-provides-first-product-details-for-EPP-ARM-based-devices

so - does that change things at all? :slight_smile:

i assumed that it would be possible to push other sections of the
gallium3d code through the LLVM wringer (so to speak). not just the
shader program. i've seen papers for example - someone won 3rd prize
from a competition by xilinx, he was a Seoul University student,
managed to implement parts of OpenGL ES 1.1 in an FPGA, by porting
MesaGL to it. got fair performance, too. i always wondered what
happened to his code, and if he would be required to comply with the
GPL / LGPL...

anyway, yes: what's possible, and where can people find out more
about how gallium3d uses LLVM? and (for those people not familiar
with 3D), why is the shader program not "static" i.e. why is a
compiler needed at runtime at _all_? (if there's an answer already
somewhere on a wiki, already, that'd be great).

and: would moving this compiler onto the FPGA (so that it interprets
the shader program), making it an interpreter instead, be a viable
option? just throwing ideas out, here. btw i'm familiar with the
dynamic architecture of 3D engines, but many people who might be
reading this (from the archives) are not. i've seen the hardware
block diagram for the PowerVR SGX engine, for example and it's...
interesting, to say the least :slight_smile:

l.

Luke Kenneth Casson Leighton wrote:

The way in which Gallium3D targets LLVM, is that it waits until it receives
the shader program from the application, then compiles that down to LLVM IR.
That's too late to start synthesizing hardware (unless you're planning to
ship an FPGA as the graphics card, in which case reprogramming is still too
slow, and it'll be too expensive).

  nick.... the Zynq-7000 series of Dual-Core Cortex A9 800mhz 28nm CPUs
have an on-board Series 7 Artix-7 or Kinect-7 FPGA (depending on the
Zynq range). and that's on the same silicon IC :slight_smile: so, price is not
an object any more [assuming reasonable volume].

  here's the eetimes article which mentions that the low-end version of
the 7000 series will be under $USD 15 in mass-volume:
  http://www.eetimes.com/electronics-news/4213637/Xilinx-provides-first-product-details-for-EPP-ARM-based-devices

  so - does that change things at all? :slight_smile:

No, because that doesn't have:
  - nearly enough gates. Recall that a modern GPU has more gates than a modern CPU, so you're orders of magnitude away.
  - quite enough I/O bandwidth. Assuming off-chip TMDS/LVDS (sensible, given that neither the ARM core nor the FPGA have a high enough clock rate), the limiting I/O bandwidth is between the GPU and its video memory. That product claims it can do DDR3, which is not quite the same as GDDR5.

You could try to trade-off between FPGA and the ARM core, but the ARM is only running at 800MHz. Maybe it's possible to get competitive performance, but it doesn't sound like a promising start.

  i assumed that it would be possible to push other sections of the
gallium3d code through the LLVM wringer (so to speak). not just the
shader program. i've seen papers for example - someone won 3rd prize
from a competition by xilinx, he was a Seoul University student,
managed to implement parts of OpenGL ES 1.1 in an FPGA, by porting
MesaGL to it. got fair performance, too. i always wondered what
happened to his code, and if he would be required to comply with the
GPL / LGPL...

Wow, that must've been a lot of work. This is what I was alluding to in my second paragraph of the previous email, except that I didn't realize someone had actually done it.

Of course, OpenGL ES 1.1 is still fixed-function hardware. That's a much easier problem, and not useful beyond current-gen cell-phones.

  anyway, yes: what's possible, and where can people find out more
about how gallium3d uses LLVM?

Ask the Mesa/Gallium folks, we really just get the occasional bug report. Personally, I follow zrusin.blogspot.com for my Gallium3d news.

   and (for those people not familiar

with 3D), why is the shader program not "static" i.e. why is a
compiler needed at runtime at _all_? (if there's an answer already
somewhere on a wiki, already, that'd be great).

  and: would moving this compiler onto the FPGA (so that it interprets
the shader program), making it an interpreter instead, be a viable
option? just throwing ideas out, here.

I'm sure there's many ways to produce an open graphics chip and card, and I'm sure there's even ways to use LLVM to simplify the hardware design, and maybe even the drivers. The point I'm trying to make is that This Stuff Is Really Hard and there's no prepackaged answer that us folks on a mailing list are going to be able to give you based on a list of products. It's certainly not a situation of "push a button" and compile a new GPU. There's a lot of avenues to try, and I don't know enough to tell you which one to pursue.

If you have questions about LLVM itself, we'll happily do our best to answer them!

Nick

GDDR5 is fairly recent. DDR3 is about equivalent to GDDR3, which came out less than a decade ago.
So it depends on whether you're trying to get top-notch performance or just acceptable performance, IMO.
I am in agreement with the rest of what you said here.

The way in which Gallium3D targets LLVM, is that it waits until it
receives
the shader program from the application, then compiles that down to LLVM
IR.

nick.... the Zynq-7000 series of Dual-Core Cortex A9 800mhz 28nm CPUs
have an on-board Series 7 Artix-7 or Kinect-7 FPGA (depending on the

so - does that change things at all? :slight_smile:

No, because that doesn't have:
- nearly enough gates. Recall that a modern GPU has more gates than a
modern CPU, so you're orders of magnitude away.
- quite enough I/O bandwidth. Assuming off-chip TMDS/LVDS (sensible, given
that neither the ARM core nor the FPGA have a high enough clock rate),

well the Series 7 has 6.6gb/sec adjustable Serial Transceivers, and
there are cases where people have actually implemented DVI / HDMI with
that. but yes, a TFP410 would be a goood idea :slight_smile:

the
limiting I/O bandwidth is between the GPU and its video memory. That product
claims it can do DDR3, which is not quite the same as GDDR5.

ahh, given that OGP is creating a Graphics Card with a PCI (33mhz)
bus, 256mb DDR2 RAM and a Spartan 3 (4000), i think that trying to set
sights on "the absolute latest and greatest in GPU Technology" is a
leeetle ambitious :slight_smile:

but, you know what's just incredible? that this debate can be had
*at all*. if this CPU didn't exist, there wouldn't _be_ the
possibility of an affordable platform on which to even *try* to create
a GPU engine.

You could try to trade-off between FPGA and the ARM core, but the ARM is
only running at 800MHz. Maybe it's possible to get competitive performance,
but it doesn't sound like a promising start.

the goal isn't to get competitive performance - the goal is to get
from way behind even the starting line. i'd settle for "good enough"
performance. gimme 1280x720, 25fps, 15-bit colour, and i'd be happy.

shader program. i've seen papers for example - someone won 3rd prize
from a competition by xilinx, he was a Seoul University student,
managed to implement parts of OpenGL ES 1.1 in an FPGA, by porting

Wow, that must've been a lot of work. This is what I was alluding to in my
second paragraph of the previous email, except that I didn't realize someone
had actually done it.

Of course, OpenGL ES 1.1 is still fixed-function hardware. That's a much
easier problem, and not useful beyond current-gen cell-phones.

rright. ok. understood.

anyway, yes: what's possible, and where can people find out more
about how gallium3d uses LLVM?

Ask the Mesa/Gallium folks, we really just get the occasional bug report.
Personally, I follow zrusin.blogspot.com for my Gallium3d news.

ok.

and (for those people not familiar

with 3D), why is the shader program not "static" i.e. why is a
compiler needed at runtime at _all_? (if there's an answer already
somewhere on a wiki, already, that'd be great).

and: would moving this compiler onto the FPGA (so that it interprets
the shader program), making it an interpreter instead, be a viable
option? just throwing ideas out, here.

I'm sure there's many ways to produce an open graphics chip and card, and
I'm sure there's even ways to use LLVM to simplify the hardware design, and
maybe even the drivers. The point I'm trying to make is that This Stuff Is
Really Hard and there's no prepackaged answer that us folks on a mailing
list are going to be able to give you based on a list of products. It's
certainly not a situation of "push a button" and compile a new GPU.

shaame :slight_smile:

There's
a lot of avenues to try, and I don't know enough to tell you which one to
pursue.

and... you know what? at least some affordable re-programmable
hardware gives a wider audience of potential free software developers
the opportunity to even try.

even if the Zynq-7000 in its current form isn't good enough to be
"good enough" for 3D, there will be new versions down the line. 22nm
and so on.

If you have questions about LLVM itself, we'll happily do our best to answer
them!

thanks nick

l.

http://www.altera.com/literature/dc/1.8-2005_Korea_3rd_Donga-PusanU-web.pdf

found it. donga not seoul, and altera not xilinx oh well :slight_smile: but
yes, probably GPL violations there *sigh* _when_ will people get with
the picture?? :slight_smile:

l.