tiny-tpu

March 23, 2026 · View on GitHub

A minimal tensor processing unit (TPU), reinvented from Google's TPU V2 and V1.

https://github.com/user-attachments/assets/b5d6aefe-4250-4c6d-866e-65d519e4de74

https://github.com/user-attachments/assets/2a501151-bca8-4458-8015-81520375781c

Table of Contents

Project Folders

  • mnist_demo/: DE1-SoC MNIST demo with organized FPGA entry points in fpga/, JTAG host utilities in host/, checked-in and generated model data in data/model/, and generated outputs collected under artifacts/.
  • tiny-tpu-hardened/: Hardened TPU variant with ASIC-ready RTL in rtl/, OpenLane inputs in openlane/, and focused regression benches in sim/.
  • xor_demo/: Compact DE1-SoC XOR demo with Intel Quartus assets in fpga/, reusable TPU RTL in rtl/, board-level simulation in sim/, and preserved Quartus outputs in artifacts/.

Architecture

TPU Architecture

Processing Element (PE)

  • Function: Performs a multiply-accumulate operation every clock cycle
  • Data Flow:
    • Incoming data is multiplied by a stored weight and added to an incoming partial sum to produce an output sum
    • Incoming data also passes through to the next element for propagation across the array

Systolic Array

  • Architecture: A 2D grid of processing elements
  • Data Movement:
    • Input values flow horizontally across the array
    • Partial sums flow vertically down the array
    • Weights remain fixed within each processing element during computation
  • Input Preprocessing:
    • Input matrices are rotated 90 degrees (implemented in hardware)
    • Inputs are staggered for correct computation in the systolic array
    • Weight matrices are transposed and staggered to align with mathematical formulas

Vector Processing Unit (VPU)

  • Performs element-wise operations after the systolic array
  • Control: Module selection depends on the computation stage
  • Modules (pipelined):
    1. Bias addition
    2. Leaky ReLU activation function
    3. MSE loss
    4. Leaky ReLU derivative

Unified Buffer (UB)

  • Dual-port memory for storing intermediate values
  • Stored Data:
    • Input matrices
    • Weight matrices
    • Bias vectors
    • Post-activation values for backpropagation
    • Activation leak factors
    • Inverse batch size constant for MSE backpropagation
  • Interface:
    • Two read and two write ports per data type
    • Data is accessed by specifying a start address and count
    • Reads can occur continuously in the background until the requested count is reached

Control Unit

  • Instruction width: 94 bits
  • See Instruction Set section below for more information.

Instruction Set

Our ISA is 94 bits wide. The full image is available in the images/ folder.

Our ISA defines all necessary signals for transferring data and interacting with our TPU. The implementation of the control unit (which reads instructions) can be found at src/control_unit.sv.

The instruction bus is 94 bits wide ([93:0]) and is divided into fields that directly control subsystems.

Bits [0–4]: 1-bit Control Signals

BitSignalMeaningExample
0sys_switch_inSystem mode switch (general-purpose "on/off" CU)1 = system active, 0 = idle
1ub_rd_start_inStart UB (Unified Buffer) read transaction1 = trigger read, 0 = no read
2ub_rd_transposeUB read transpose mode1 = transpose, 0 = normal
3ub_wr_host_valid_in_1Host write channel 1 valid flag1 = write valid, 0 = not valid
4ub_wr_host_valid_in_2Host write channel 2 valid flag1 = write valid, 0 = not valid

Bits [6:5]: UB Read Column Size (2-bit)

FieldSignalMeaningExample
[6:5]ub_rd_col_sizeNumber of columns to read00=0, 01=1, 10=2, 11=3

Bits [14:7]: UB Read Row Size (8-bit)

FieldSignalMeaningExample
[14:7]ub_rd_row_sizeNumber of rows to read (0–255)0x08 = read 8 rows

Bits [22:15]: UB Read Address (8-bit)

FieldSignalMeaningExample
[22:15]ub_rd_addr_inUB read address (0–255)0x10 = read bank 16

Bits [25:23]: UB Pointer Select (3-bit)

FieldSignalMeaningExample
[25:23]ub_ptr_selSelects UB pointer3’b001 = route read ptr to bias module in VPU

Bits [41:26]: UB Write Host Data In 1 (16-bit, Fixed-Point)

FieldSignalMeaningExample
[41:26]ub_wr_host_data_in_1First host write word0xABCD

Bits [57:42]: UB Write Host Data In 2 (16-bit, Fixed-Point)

FieldSignalMeaningExample
[57:42]ub_wr_host_data_in_2Second host write word0x1234

Bits [61:58]: VPU Data Pathway (4-bit)

FieldSignalMeaningExample
[61:58]vpu_data_pathwayRouting of data in VPU0001=bias + relu routing

Bits [77:62]: Inverse Batch Size × 2 (16-bit, Fixed-Point)

FieldSignalMeaningExample
[77:62]inv_batch_size_times_two_inPrecomputed scaling factor (2/batch)0x0010 = (2/32)

Bits [93:78]: VPU Leak Factor (16-bit, Fixed-Point)

FieldSignalMeaningExample
[93:78]vpu_leak_factor_inLeak factor for activation (e.g., Leaky ReLU)0x00A0 = 0.625

Example Instruction Sequence

Instructions are directly loaded into an instruction buffer on the chip from a testbench file.

  • See tests/test_tpu.py for our forward and backward pass instruction sequence
  • See the Setup section on how to run this testbench

Future Steps

  1. Compiler for this instruction set
  2. Scaling TPU to larger dimensions (256×256 or 512×512)

Setup

We are open source and appreciate any contributions! Here is our workflow and steps to set up our development environment:

MacOS Specific

  1. Create a virtual environment and run: 
    pip install cocotb
  2. Install iverilog using Homebrew: 
    brew install iverilog
  3. Build gtkwave FROM SOURCE (important: other installation methods currently do not work)

Ubuntu/Linux Specific

  1. Create a virtual environment and run: 
    pip install cocotb
  2. Install gtkwave: 
    sudo apt install gtkwave
  3. Install iverilog: 
    sudo apt install iverilog

Adding Modules

Follow these steps to add a new module to the project:

1. Create the Module File

Add your new module file <MODULE_NAME>.sv in the src/ directory.

2. Create the Dump File

Create dump_<MODULE_NAME>.sv in the test/ directory with the following code:

module dump();
initial begin
  $dumpfile("waveforms/<MODULE_NAME>.vcd");
  $dumpvars(0, <MODULE_NAME>); 
end
endmodule

3. Creating Tests

Create test_<MODULE_NAME>.py in the test/ directory.

4. Makefile Updates

Add your module to the SOURCES variable and create a test target:

test_<MODULE_NAME>: $(SIM_BUILD_DIR)
	$(IVERILOG) -o $(SIM_VVP) -s <MODULE_NAME> -s dump -g2012 $(SOURCES) test/dump_<MODULE_NAME>.sv
	PYTHONOPTIMIZE=$(NOASSERT) MODULE=test_<MODULE_NAME> $(VVP) -M $(COCOTB_LIBS) -m libcocotbvpi_icarus $(SIM_VVP)
	! grep failure results.xml
	mv <MODULE_NAME>.vcd waveforms/ 2>/dev/null || true

5. View Waveforms

Run the following command to view the generated waveforms:

gtkwave waveforms/<MODULE_NAME>.vcd

Makefile Commands

Run tests:

make test_<MODULE_NAME>

View waveforms:

gtkwave waveforms/<MODULE_NAME>.vcd

Or use the shorthand:

make show_<MODULE_NAME>

GTKWwave Setup

  1. Right-click all signals
  2. Navigate to: Data Format → Fixed Point Shift → Specify
  3. Enter 8 and click OK
  4. Set: Data Format → Signed Decimal
  5. Enable: Data Format → Fixed Point Shift → ON

What is a .gtkw File?

A .gtkw file stores the signal configuration for make show_<MODULE_NAME>. You only need to save it once after running:

gtkwave waveforms/<MODULE_NAME>.vcd

Motivation

The details of TPU architecture are closed source, as is most of chip design. We want this resource to be the ultimate guide to breaking into building chip accelerators for all levels of technical expertise — even if you just learned high school math and only know y = mx + b.

Before this project, none of us had professional experience in hardware architecture/design. We started this ambitious project as a dedicated group wanting to break into hardware design. We've collectively gained significant design experience from this project.

We hope that the inventive nature of the article at tinytpu.com, this README, and the code in this repository will help you walk through our steps and learn how to approach problems with an inventive mindset.