Introduction to PYNQ

Python with an FPGA? No Way

The questions below are due on Friday September 13, 2024; 04:59:00 PM.
 
You are not logged in.

Please Log In for full access to the web site.
Note that this link will take you to an external site (https://shimmer.mit.edu) to authenticate, and then you will be redirected back to this page.

In this lab we'll build up a basic "full" Zynq system that contains both the Processing System (PS) and the Programmable Logic (PL). The final result should be something similar to the video below, which shows a system which takes both hardware buttons and computational inputs. It'll make more sense when you're actually building towards

The video probably just looks like a crazy person making LEDs flash, but trust me, this is a system using both software and hardware inputs together generate an output.

Prior to Getting Started

Because we'll be working on both the Processing Systemt ("PS") and with the Programmable Logic ("PL"), we're going to need to make sure you have the system all set up in regards to the Pynq Board Image that will go on the SD card. Please grab a microSD Card that is pre-flashed with the appropriate PYNQ framework on it (there should be a labeled box at the front of class or ask Joe or Kiran). This microSD card will be yours for at least the first few weeks so you should keep it with you and remove it from the Pynq board when done (since we'll be keeping the boards in the lab room for now. Don't put it in yet. We'll do that later.

You can worry about booting the Pynq up later. Let's do some designing first.

Getting Started

We need to open up Vivado in Project Mode, this means using the Graphical User Interface. Deep breath. Previously in 6.111 we built our projects in non-project mode which actually resulted in slightly longer build times since the system wouldn't cache the design in stages, but did allow us to keep a much, much lighter-weight file structure. For the sake of working with the Zynq we will be working in Project Mode. For right now, just roll with it. Later in the semester, in particular before we hit final projects, I'll give you some tcl scripts to help you work with Project mode in a way that allows for version control.

  1. Launch Vivado from a terminal by just running vivado with no other arguments. (if that doesn't work for some reason, the binary is under /tools/Xilinx/Vivado/2024.1/bin/vivado on our lab machines. Either way the GUI will open up.
  2. Click Next on Create a New Project
  3. Name it something you'll remember like led_controller_1 or something and click Next
  4. Select it as an RTL Project and click Next
  5. Don't worry about adding Verilog sources right now so click Next
  6. Add the `base.xdc file found here (and make sure to specify "Copy Constraints File Into Project")
  7. When prompted to choose a device do one of the following (and after that click Next):
    • Go to the boards page and choose the Pynq Z2 board (if that file is installed...it'll probably show an image of the Pynq Board...and it should be installed if you're on a lab machine...if you're working on your own personal Vivado install, you'll want to add the board files. Email me and I'll show you how.). This is the preferred method since it will associate some important other constraints with your project that have to do with timing issues.
      • if the board isn't already there: first click the "refresh" button, then search the boards for "pynq", and click the "download" looking icon under the "Status" column. This should make the preview picture of the right board load in.
  8. Click Finish

Some stuff will fly by and then boom, you'll be in the project mode. Right now there's only one file, your xdc, which you can see in the upper left side Sources manager. We need to add more. Let's do that.

Starting Our Block Diagram

We want to eventually build this block diagram of our system:

block diagram overall

Block Diagram of our system.

OK so if you're coming from 6.111/6.205, this would be the point where we create/add a top-level Verilog file. (we'll eventually do that but we'll have Vivado generate it automatically from what we're about to do here). We'll instead go the block diagram route for now because AMD/Xilinx Vivado really, really wants you to use the graphical pipeline for doing anything with Zynq products. It is not ideal, but it isn't as bad as I made it sound in lecture. Just don't expect amazing things from it.

Activating and Configuring the Processor

We first need to make a design entry point for the processor on the system....something that let's us customize and modify the Processing System ("PS") and will then allow us to link things we design in the Programmable Logic (the FPGA part) (the "PL") to it.

  1. Under IP Integrator on the Left side, click on Create Block Diagram. Call it something you want (I usually just do design_1, but you do you). Keep file locations as the defaults and/or locals

  2. A blank Diagram window should appear to the right. It is blank and we want to eventually fill it so that it will look like the first image above (our final system for this lab). To do that we're going to add IP and modules. We're essentially doing what we've always done in 6.205 with straight-up Verilog, but instead we'll be doing it graphically and this can help in a lot of ways.

  3. Add a piece of IP to the design by clicking on the Add IP button (or press control-I).

  4. Search for and add the ZYNQ7 PROCESSING SYSTEM. Click on this.

  5. After it is added, you should almost immediately see a green option window at the top that says Run Block Automation.

    • Often times Vivado will suggest things to connect for you. Be careful with this since it is stupid sometimes, but really helpful other times. This first one, we can just trust it, so let it do its thing. Click on it.
    • In the window that pops up, it'll tell you what it is trying to auto-connect. Always review this! It'll say something about making DDR and Default IO external for us. This is good. Let it do that. Click OK
    • Some new wires should appear as well as some External Interfaces labeled DDR and FIXED_IO which correspond to output pins (a subset of the output pins can be accessed by the PL, and a subset can be accessed by the PS)
  6. We now need to customize the Zynq PS for our purposes. Big picture, for this first lab we really want the ability to just "share some bits" between the PL and the PS. There are many ways to do this from very structured ways using AXI ports, to large-data-throughput ways like direct memory access. For now we'll just do a relatively low-throughput connection (GPIO).

To do this double click on the module. First off we're going to disable the AXI ports that connect the PS to the PL directly. (We'll use those later in other labs or in projects). Go to the PS-PL configuration tab and make sure no options are checked (including under any submenus). They will probably already be unchecked, but double check. Sometimes, some default setting gets wonkky. It should look similar to below:

no_periph

  1. Next we want to enable some "General Purpose Input Output" (GPIO) pins on the PS. To do this click on the Peripheral I/O Pins tab and scroll to the bottom, selecting GPIO MIO and GPIO EMIO like shown below (you'll note the pins will turn green indicating they are in use) At the same time make sure to disable all other uses for these pins (Flash SPI, UART, etc.)

io_setup

  1. Finally let's enable a clock of a particular frequency. Zynq systems have a number of clocks that run in the PL, but which are configurable and controllable from the PS. This allows the software side of things to vary the speed of logic implemented as needed (and this can be done with very simple python API calls since it is really just nudging a single MMIO mapped register). Some more info here. Go to the Clock Configuration tab and under PL Fabric Clocks, select FCLK_CLK0 and set it to generate a 50 MHz frequency like shown:

clock_setup

  1. When the above three things have been done, click on OK and the Zynq system will update itself (some pins will appear, others will dissapear). It should look like the following when done. Make sure it matches, since unused pins will cause issues:

ending_stage_1

Adding Our Own Interfaces

We now want to interface to some outside connections that the PL is connected to. In particular we want to interface to the buttons and to the LEDs on the PYNQ board.

  1. Open up your base.xdc file (find it under Sources>Constraints) and make sure that everything is commented out except for the set of lines referring to the LEDs and the buttons. Depending on the version of your XDC, they may have different names. Plurality and Capitalization matter so pay attention to what they are called. Your XDC should look similar to below (make sure to save the file):

xdc

  1. Next, go back to the block diagram, and Right Click > Create Port or press Control-K. This will bring up the option to create inputs and outputs to the PL. Create two ports, an input for the buttons and an output for the LEDs. Should make sense:
    • Create one with the exact name of your LED pins. Make it an Output and a Vector from 3 to 0 (since there are four LEDs)
    • Next create another port with the exact name of your button pins. Make it an Input and a Vector from 3 to 0 as well.

pin_naming

Making an output port

When all finished, you should have an input and an output port.

As you may remember from 6.205/111, the XDC file maps the cryptically named pins on the FPGA (names like D19 and R14 which express position in the ball grid array) to meaningful names that we can use in our designing. When building the bit file up, Each active pin also requires some setup/interfacing to be built around it so make sure to only uncomment pins that you're using otherwise you'll make your build times unnecessarily long (and they are already long as it is).

Integrating Verilog Modules

We now want to create two modules in Verilog. The first module will be a pulse generator (it takes in a clock and generates a periodic event from it. Nothing crazy. Should be EZ to do since you've done 6.205...in fact we'll just give it to you as a warmup).

To make a new module go to PROJECT MANAGER >> Add Sources >> Add or Create Design Sources, and then Create File. Name it whatever you want (maybe pulse_maker.sv is a good one, just keep it local to your project and avoid spaces in the name, otherwise it'll be hell. Finish and then move on. Vivado will then try to "help" you by allowing you to declare the Verilog module you want to write inside this creation GUI. Just skip this part unless you like wasting time. You can type it out way faster.

Here's the source...assuming the clk is 100 MHz (not necessarily true, this will fire 5 times per second)

`timescale 1ns / 1ps
`default_nettype none
//will fire a pulse once every 20 million clock cycles
//feel free to change/modify
module pulse_maker( input wire clk,
                    input wire rst,
                    output logic pulse);
    logic[31:0] counter;
    localparam PERIOD = 20_000_000;
    always_ff @(posedge clk) begin
        if(!rst) begin //active low reset just for fun
            counter<=32'd0;
        end else begin
            if(counter==PERIOD-1)
                counter<=32'd0;
            else
                counter<=counter+1;
        end
    end
    assign pulse = (counter == PERIOD-1); //feel free to change this later.
endmodule

`default_nettype wire

Add this code into your SystemVerilog file. Save your code.

Now here is an annoying part. Vivado's Block Diagram Editor will not permit the integration of SystemVerilog files into it directly. I honestly don't know why. This was an issue six years ago and I thought it was a feature they just didn't get to yet, but they've never gotten to it and they don't seem to have any intention of supporting it. In their defense there are probably some legit reasons for it given that supporting SystemVerilog means having to support a lot of other interesting things like interfaces so I do kinda get it, but it is annoying. Nevertheless, how do we integrate SystemVerilog into the project? By wrapping it in Verilog. No joke. That doesn't even seem like it should be legal tbh. It is like when gmail blocks you from sending a python file because it thinks it is malicious so you just change the extension to .txt and it gets right through the filter.

Make a new file and call it pulse_maker_w.v. In that, make a "pass-through" module that literally has the same inputs and outputs as the actual pulse maker module you wrote above. This wrapper module should have an instance of the pulse_maker in it. It'll look like this:

`timescale 1ns / 1ps
`default_nettype none
module pulse_maker_w(   input wire clk,
                        input wire rst,
                        output wire pulse); //non-systemverilog verilog does not have logics only wires.

    //instance of pulse maker:
    pulse_maker mpm (   .clk(clk),
                        .rst(rst),
                        .pulse(pulse));
endmodule

`default_nettype wire

OK Now this is what we'll integrate into the design.

Assuming you have a working module, we now need to integrate it into the block diagram. To do this:

  1. Right click on the background of your block diagram and click on Add Module. Select your pulse_maker_w module and it will appear!
  2. Drag the module to somewhere convenient

At this point you can potentially run some connection automation (it will likely prompt you at the top). If you do that, the system will attempt to wire the fabric clock from the Zynq processor to the pulse_maker_w input. In the process it'll add a clocked reset module. You can either let it do this or skip the clocked reset module and wire the Zynq Processor clock to the pulse maker clock and its reset to the reset of the pulse maker manually. It is up to you. For this lab it won't matter. If you don't let it automate/do it manually, you'll get a warning about an asynchronous/non-clocked reset signal, but you can ignore it. In more complicated designs you should probably let it do its thing here.

pin_naming

Included reset module (not needed, but keep if you want)

pin_naming

Wiring this thing up without a reset.

  1. If you don't let Vivado automatically connect your pulse maker, connect the fabric clock to your clock input and the Zynq reset to the reset on your pulse maker module. You can connect stuff by hovering over the port/input/output of interest on a module and a little pencil symbol will appear. Click and drag/move your mouse to where you want to connect it. It'll snap and auto-route for you.

LED Controller

We want to make another module now that will sort of strobe the four LEDs using our divided clock and go either left or right (or stop them) based on three different control signals:

  • One signal (Stop signal) will come from the Zynq PS. We'll control this through Python
  • One signal (Go left signal) will come from the button connected to the PL
  • One signal (Go right signal) will come from a second button connected to the PL

You can include the module's Verilog in the same file you put your pulse_maker module in. (Is up to you...honestly I'd avoid it). A starter skeleton is shown below, but you need to write this one. Watch the video again for how we want it to work.

module  led_controller (input wire clk,
            input wire en,
            input wire go_up,
            input wire go_down,
            input wire stop,
            output logic[3:0] q);

//your Verilog here

endmodule

Once you get this module working, do the same thing you did for your pulse_maker up above, including making a Verilog wrapper for it. Include it in the block diagram, and wire its four bit output to the LED external interface. This should be no problem.

You will run into a problem with the inputs on this module, however. Each input pin on the this module is one-wide whereas the buttons come in as a four-wide vector, and the GPIO out from the Zynq processor is a 64-wide vector. This will cause issues. Damn. Well there is a solution. We could actually just write some Verilog modules to handle this. That is one solution. However there is alreayd IP for this believe it or not...they're called slice modules. We can use some slice modules to select/de-mux down. To do this go to the Add IP window and search for Slice. Add one and double click on it. We will need three Slices:

  • For the one that slices the Zynq PS GPIO, make its input width (Din) be 64. Then make both Din From and Din Down To have a value of 2. This is telling the slicer to take in all 64 values and only put out value on pin 2.

pin_sizing

  • Make another slicer that slices the four button inputs down to 0 to 0.
  • Make another slider that slices the four button inputs down to 1 to 1.

To connect to the GPIO pins, hover over them and a down-down arrow symbol will appear. Click this to expand out the GPIO OUTPUT, INPUT, and T pins (used with high-impedance paths). Then connect to Output!

You can name your Slicer (or any module) through the Block Properties box that should come up when you click on it. Name your one coming from the GPIO thing, stop.

Run the appropriate wires/connections between all these modules. When completed you should have a system that looks like the following (shown earlier, but again here for clarity)

block_diagram_overall

Double check that things match up! If you let Vivado wire up your pulse_maker automatically, you'll also have a reset module in there. No worries either way.

Note if you already have Verilog files written somewhere else, you could be importing them at this point. Regardless, makes sure you always keep copies of the actual files in your project directory and just let Vivado manage it. Otherwise, you'll maybe get weird errors/permission issues.

Putting It All Together

OK if everything is good so far, you can do a couple things:

  1. Click on the Optimize Routing button (search for it...it is like a twirly arrow and a right-angle-step wire) in the top bar of the Diagram window. This will clean up everything for you (and generally does a good job at it).
  2. Then click on Validate Design. (check mark symbol) This will do a first-pass (very surface-level) sanity check of your system and flag any potential problems (unconnected wires, incompatible pins, etc). Depending on if you added the reset module or not, you'll get one warning about an asynchronous reset. For this lab, don't worry. If you get other warnings/issues, read them through. They will be telling you information! Adjust your design accordingly.
  3. If all is good, under the Sources menu, right click on the block diagram file (the .bd file which has a symbol that looks like a stack of gold bars), and click on Generate Output Products. This will generate some important linker files, including the important hardware handoff file we'll need to use in a bit.
  4. In addition, if still no issues, under the Sources menu, right click on the block diagram file (the .bd file which has a symbol that looks like a stack of gold bars), and click on Create HDL Wrapper. Let Vivado manage this thing for you. In the future maybe you will manage it yourself, but we're not there yet emotionally. This will create a high-level Verilog file for us/converting the block diagram into it.
  5. There should now be a design_1_wrapper thing in the sources tab on the left. Make sure that is BOLD with a three-box-green-on-top-box symbol. If not, right click on it, and click on "Set as Top". This is how we tell Vivado that this is the entry point of our project. Previously when doing tcl scripts, there'd be a line that specifies this, but we need to do it here. Wait for a bit while things flash and it should appear bold now. If you double click to expand it, inside there will be something named design1.v. This is actually the source file for you design. This is a standard Verilog file made from your block diagram. It should, for the most part make sense, and if not, at least be familiar looking. This really is the goal/dream/intention of this block diagram thing...automating a bunch of annoying module instantiations....now back to the journeykkkk
  6. If still no errors pop up, then go and click on Generate Bit Stream. This will start the whole build process of Synthesis, then Implementation, and then Generating the Bitstream. Depending on your computer, this could take a couple minutes for this project (and potentially longer if you forgot to comment out unused things in your XDC). Errors could appear at any point in this build process so pay attention to the error logs and ask for help if needed. They will appear in the bottom terminal thing. Make sure to have Errors and Critical Warnings and Warnings visible. As we've seen in 6.205...Errors are really bad. Critical Warnings are usually bad, but sometimes fine. Warnings are often ok, but can be symptomatic of issues.
  7. Cross your fingers while it runs.
  8. If errors come up as it builds, note what they are and either ask for help, or look them up.
  9. When the Build is complete, MAKE SURE YOUR BLOCK DIAGRAM DESIGN IS OPEN and then go up to File >> Export >> Export Block Design. This will generate a .hwh file with the name of your block diagram design. We'll need it, and the bit file you just generated in the next section.

Moving it to the Zynq PS

If everything went ok in the previous section we need to move two files to your Zynq over the network. These files will live inside of your project folder for what you were just building. Assuming your project was called cool_project and your block diagram was called design_1 you'll find:

  • design_1.hwh in cool_project/cool_project.gen/sources_1/bd/design_1/hw_handoff/design_1.hwh
  • design_1_wrapper.bit in cool_project/cool_project.runs/impl_1/design_1_wrapper.bit

We need to copy both of these files over to the Zynq file system now. So this brings us to actually powering up the Zynq board. Let's get started.

Getting the Zynq On

You should have a microSD card that will be your Pynq OS.

  1. Make sure the Pynq board is powered off.
  2. Insert your microSD card.
  3. Power the Pynq on. It will take maybe a minute or so. When done booting, it will flash some on board LEDs and the "DONE" LED will be green.

We now need to get onto the Pynq board. You'll notice it is connected with an ethernet cable. That gives it network access and it should be accessible in that manner. We've set up mini router networks for all Zynq stations in the lab. In those stations, the lab computer is also on the network (and if you'd like your computer can get on the network too by using an ethernet adapter. We can't do wifi since all the little wifi networks will jam eachother so wired it is going to be).

Anyways from either the lab computer or your laptop on the network you should be able to go to your router at your station's IP address (like 192.168.0.1) and under there you should be able to see what IP address your Pynq board has taken. Often times it'll be 192.168.0.101.

In a web browser, visit that IP address. It'll prompt you with a login. Username and password are both xilinx. Afterwards you'll get something like this:

pynq_site

This is a jupyter environment hosted on the processign cores of the Zynq board. Kinda cool. Anyways inside of this let's make a new folder for this project. You can see in the image above I made one and called it led_controller. Move into there and create a new notebook. We'll use that in a minute.

Also in that folder, we need to upload the two important files we talked about earlier. There should be an upload button. If you're doing this on the machine that did the builds, you can navigate in that to get the two files up. I renamed them led_c.hwh and led_c.bit, but you can name them whatever once they're inside. Just make sure they have the same base name!!! That is important.

As an alternative, you can also just scp these files up from the lab machine to the Zynq OS no problem too. Doing something like this from the lab machine will put the hwh file into the base jupyter notebooks folder on the zynq os (note password to do this is xilinx).

scp design_1.hwh xilinx@192.168.0.101:~/jupyter_notebooks/

I know this can be confusing if you've not done this before (I've been there, please believe). Feel free to ask for help about this or post on Piazza or something. Once you get the hang of it, it is fine, but before it can seem daunting.

If these got placed on the machine with no errors we now need to go to the Jupyter Notebook to do something with them.

Running it on the Zynq PS

We can interact with our system via Python using two different methods. The first is you can just ssh into the Pynq board, and write a Python file and run it right from the terminal. If you're comfortable doing that, then great. I tend to do this since I wasn't raised on Jupyter notebooks, but I gotta say I do see the point of Jupyter notebooks and that approach so totally go with that if you'd like. The Jupyter notebooks also allow an easy way to get some graphical feedback.

If you do the Jupyter approach, in a web browser (on the same network as your router that the Zynq is connected to), type in the IP address of your board. It'll prompt you to log in. Again, the password is xilinx. When there, feel free to make a new directory and/or move folders and files around. I made a directory called led_counter and put the ledc.hwh and ledc.bit file into it. Then I moved into that directory and created a new Python3 notebook.

python_notebook_overall

Into that notebook I put the following chunks of code and ran them in sequence:

python_notebook

The first one is the following:

from pynq import Overlay
ol = Overlay("./led_c.bit")

This code imports the Overlay library which we then use to load the bit file we just generated. When you do this, the library looks for the similarly named hwh file, which should be located in the same directory (you moved that in right?). When that gets done and checked, you should be good to move on. If you run this blob, you should see your LEDs come on (but probably not be visible sweeping left or right)...or they might be just barely visibly flickering...the problem is our clock is still too high in frequency. We'll fix that in a little bit.

Next I upload the Python API to interface with the GPIO pins. I set pin 2 (remember we Sliced that one down to be our "stop") signal

from pynq import GPIO
stop = GPIO(GPIO.get_gpio_pin(2),'out')

I next added a line to make sure the "stop" signal is off for right now.

stop.write(0)

Finally we can change the frequency of our fabric clocks from Python. This is really nice since it can let you get some debugging capabilities without having to rebuild your file.

from pynq import Clocks
Clocks.fclk0_mhz = 100
print(f"FCLK0: {Clocks.fclk0_mhz:.6f}MHz")

These lines of code should ensure a visible LED chase phenomenon based off of the button presses like shown in the video... If you'd like to turn off your LED chase, you can do it via Python!

stop.write(1)

Then to turn it back on (in my implementation anyways) I have to:

stop.write(0)

and then press either the left or right buttons that I designed.

OK....yeesh...And with this we're done! You should be able to interact directly with your Verilog code from Python and from your buttons to make a real-world change. This is really powerfull since it demonstrates a very basic way for information to go from your computational element (the Zynq Processor cores) down to a custom circuit living in the FPGA fabric.

There are more advanced and efficient means of sending data between these two environments which we'll cover in a future readthrough/lab thing, but for now we can just revel in this thing. Python has a lot of cool libraries...you could potentially have Python access online resources, scrape them, then send data down to the FPGA to be processed quickly, then send its result back up. Crazy.

As an extra bonus challenge, Feel free to return to your design and up some GPIO inputs on the PS (to read the buttons) if you want. GPIO API docs are here.

Please upload a zip folder of your design_1.v file (the product of all the block diagramming), your led_controller.sv file.
 No file selected

Please dump a link here to a video of your system work. If I'm in lab and you show it working, just call me over and I can save you the video part of this.

Lab initially inspired by video here