The Saturn Microarchitecture Manual

Saturn was developed with the following objectives:

Questions, bug reports, or requests for further documentation can be made to jzh@berkeley.edu.

We broadly categorize the range of deployment scenarios and workloads for programmable data-parallel systems into four domains. These categories are differentiated by power/area constraints and workload characteristics. Figure 2 outlines the four deployment domains where DLP-enabled architectures are critical, and the salient architectural and microarchitectural archetypes that have come to dominate each domain.

HPC, scientific computing, and ML training workloads are typically characterized by very long application vector lengths, exhibiting a high degree of data-level parallelism. Systems targeting these workloads are generally designed and deployed at datacenter scale, with available data-level-parallelism exploited across the many processors in a multi-node cluster. While single-core efficiency and performance remain important for this domain, scalability towards multi-core, multi-node systems is of more critical concern.

General-purpose server, desktop, and mobile workloads intersperse data-level-parallel codes with sequential, control-flow heavy non-data-parallel blocks. The implications of Amdahl’s Law dictate the design of these systems; exploiting data-level parallelism must not come at the expense of single-thread performance or thread-level-parallelism. Compared to the other domains, compiler and programmer-friendly ISAs are particularly important for these systems, as they must remain performant on the greatest diversity of workloads.

Digital signal processing (DSP) and similar domain-specialized workloads are fundamentally data-parallel algorithms. These workloads are often offloaded in modern SoCs to specialized "DSP cores" optimized towards power and area-efficient exploitation of data-level parallelism, with less consideration towards general-purpose performance. "DSP cores" usually run optimized kernels written by programmers at a low level to maximize efficiency. Large SoCs typically integrate many such "DSP cores" across various design points to provide optimized compute for different subsystems.

Applications executing on embedded devices face extreme constraints around power, area, and code size. Exploiting data-level parallelism can be critical for these workloads as well, but might come as a secondary concern compared to power and area. As these cores often provide a self-contained memory system, code and data size are more critical architectural considerations.

The requirements of the common data-parallel workload domains have induced the commercial space to converge towards several fundamental architectural and microarchitectural archetypes.

The RISC-V Vector ISA is the standard extension in RISC-V for exploiting data-level parallelism. A full discussion of the ISA design can be found in its specification [4]. This section highlights several properties of RVV that pose notable challenges to implementation or distinguish it from other vector ISAs.

Saturn’s instruction scheduling mechanism differentiates it from the relevant comparable archetypes for data-parallel microarchitectures. Fundamentally, Saturn relies on efficient dynamic scheduling of short-chime short-vectors, without relying on costly register renaming. When LMUL is short (1 or 2), vector chimes may be only 2-4 cycles long, requiring higher throughput scheduling than a long-chime machine.

Figure 3 depicts a simplified pipeline visualization of a short vector loop, consisting of a load and dependent arithmetic instruction, executing on a simplified Saturn-like short-vector datapath. In this example, each vector chime is 2 cycles.

A short-vector machine should fully saturate both the arithmetic and memory pipelines with such short vector lengths and chimes. Instruction throughput requirements are moderate, but can still be fulfilled with a modest superscalar in-order scalar core. Notably, some degree of out-of-order execution, beyond just chaining, is necessary to enable saturating both the memory and arithmetic pipelines.

Figure 4 highlights the importance of zero dead time for short-vector microarchitectures. Unlike in Figure 3, the machine in this example has 1-cycle dead time for each functional unit, perhaps due to an inefficiency in freeing structural resources as instructions are sequenced. Notably, a single cycle of dead time in a short-chime machine substantially degrades the utilizations of the datapaths, as the dead time cannot be amortized.

Figure 5 highlights the importance of limited out-of-order execution for short-vector microarchitectures like Saturn. Unlike in Figure 3, the machine in this example requires the instructions to enter the datapath in-order. Requiring strict in-order execution would substantially degrade performance for suboptimally scheduled vector code. Despite the same zero-cycle dead time, the restriction on in-order execution prevents the machine from aggressively issuing instructions to hide the latency in the M pipe.

Saturn is organized into three main components, as shown in Figure 10.

The Vector Frontend (VFU) integrates into the pipeline of the host RISC-V core. In in-order cores, the stages of the VFU align with the pre-commit pipeline stages of the host core. All vector instructions are passed into the vector frontend. The VFU performs early decode of vector instructions and checks them for the possibility of generating a fault by performing address translation of vector memory address operands. Instructions that do not fault are passed to the other vector components once they have also passed the commit stage of the scalar core.

The Vector Load-Store Unit (VLSU) performs vector address generation and memory access. Inflight vector memory instructions are tracked in the vector load-instruction-queue (VLIQ) and store-instruction-queue (VSIQ). The load/store paths within the VLSU execute independently and communicate with the VU through load-response and store-data ports.

The Vector Datapath (VU) contains instruction issue queues (VIQs), vector sequencers (VXS/VLS/VSS), and the vector register file (VRF), and the SIMD arithmetic functional units. The functional units (VFUs) are arranged in execution unit clusters (VEUs), where each VEU is fed by one sequencer. The sequencers schedule register read/write and issue operations into the VEUs, while interlocking on structural and data hazards. The VU is organized as a unified structure with a SIMD datapath, instead of distributing the VRF and VEUs across vector lanes. This approach is better suited for compact designs, where scalability to ultra-wide datapaths is less of a concern.

Several key principles lay the foundation for Saturn’s microarchitecture.

Saturn relies on post-commit execution of all vector instructions. That is, the VLSU and VU only receive committed instructions from the VFU. Physical addresses are passed from the VFU into the VLSU to ensure that the VLSU will never fault on memory accesses. This simplifies the microarchitecture of the VLSU and VU, as all operations in these units are effectively non-speculative. To enforce precise faults ahead of commit, Saturn’s VFU implements an aggressive, but precise mechanism that validates instructions are free-of-fault and reports any faults precisely if present.

Saturn’s microarchitecture is designed around in-order execution with many latency-insensitive interfaces. Notably, the load and store paths are designed as pipelines of blocks with latency-insensitive decoupled interfaces. The load-response and store-data interfaces into the VU are also latency-insensitive decoupled interfaces. Within the VU, each execution path (load/store/arithmetic) executes instructions in-order. The in-order execution of the load/store paths aligns with the in-order load-response and store-data ports.

Saturn is organized as a decoupled access-execute (DAE) [16] architecture, where the VLSU acts as the "access processor" and the VU acts as the "execute processor". Shallow instruction queues in the VU act as "decoupling" queues, enabling the VLSU’s load-path to run many instructions ahead of the VU. Similarly, the VLSU’s store path can run many cycles behind the VU through the decoupling enabled by the VSIQ. This approach can tolerate high memory latencies with minimal hardware cost.

Saturn still supports a limited, but sufficient capability for out-of-order execution. The load, store, and execute paths in the VU execute independently, dynamically stalling for structural and data hazards without requiring full in-order execution. Allowing dynamic "slip" between these paths naturally implies out-of-order execution. To track data hazards, all vector instructions in the VU and VLSU are tagged with a "vector age tag (VAT)". The VATs are eagerly allocated and freed, and referenced in the machine wherever the relative age of two instructions is ambiguous.

To decode vector instructions, the Saturn generator implements a decode-generator-driven methodology for vector decode. The Saturn generator tabularly describes a concise list of all vector control signals for all vector instructions. Within the generator of the VU, control signals are extracted from the pipeline stages using a generator-time query into the instruction listings. The results from this query are used to construct a smaller decode table for only the relevant signals, which is passed to a logic minimizer in Chisel that generates the actual decode circuit. This approach reduces the prevalence of hand-designed decode circuits in the VU and provides a centralized location in the generator where control signal settings can be referenced.

Saturn currently supports integration into either Rocket or Shuttle as the host scalar RISC-V core. The host cores are responsible for decoding vector instructions that must be dispatched to the VFU, in addition to updating and supplying the vtype and vl CSRs to these instructions.

Rocket and Shuttle expose nearly identical interfaces to the vector unit. The scalar pipeline dispatches vector instructions to the VFU pipeline at the X stage, immediately after scalar operands are read from the scalar register files. The scalar pipeline also must provide these instructions with up-to-date values of vtype, vl, and vstart. The M stage exposes a port through which the VFU can access the host core’s TLB and MMU. The WB stage contains interfaces for synchronizing committing or faulting vector instructions with the scalar core’s instruction scheme and stalling scalar instruction commit if necessary. The WB stage also exposes interfaces for modifying vstart and vconfig as vector instructions commit.

After entering the VFU, vector instructions first proceed through the pipelined fault checker (PFC). Instructions that the PFC cannot conservatively guarantee to be free of faults are issued to the IFC. Instructions that pass the PFC successfully can then be dispatched to the VU and VLSU after they pass the commit point.

Since vector instructions may be speculative ahead of the commit point, any vector instruction flushed by the scalar core is also flushed from the VFU. The PFC/IFC design pattern achieves the goal of making common case vector instructions fast, through the PFC, while preserving correct precise fault behavior for all vector instructions through the IFC.

The PFC and IFC share access to a single TLB port in the VFU. This TLB port would typically access the scalar core’s TLB. Future modifications to Saturn could supply a dedicated vector TLB instead.

Vector memory instructions architecturally appear to execute sequentially with the scalar loads and stores generated by the same hart. Scalar stores cannot execute while there is a pending older vector load or store to that same address. Scalar loads cannot execute while there is a pending older vector load to that same address, as doing so could violate the same-address load-load ordering axiom in RVWMO since the vector and scalar loads access different paths into the memory system. Section 4.3 discusses the mechanisms for vector-vector memory disambiguation.

The S2 stage of the PFC also receives the physical address of the current in-flight scalar load or store about to commit in the host scalar core’s W stage. This address is checked against the older inflight loads and stores in the VLIQ and VSIQ in the VLSU. On a match, a replay for the younger scalar load or store is requested.

The VLSU does not access or probe the scalar store buffers. To avoid RAW or WAW memory hazards against scalar stores in a scalar store buffer, the PFC stalls the dispatch of vector instructions in the S2 stage until the scalar store buffer is empty. We observe that this requirement has minimal impact on most vector codes, as scalar stores are rare in stripmined loops.

The micro-op presented to the VU and VLSU contains the instruction bits, scalar operands, and current vtype/vstart/vl settings for this instruction. For memory operations, this bundle also provides the physical page index of the accessed page for this instruction, since the PFC and IFC crack vector memory instructions into single-page accesses. For segmented instructions where a segment crosses a page, segstart and segend bits are additionally included in the bundle, to indicate which slice of a segment resides in the current page.

The VLSU assumes integration into a memory system that maintains coherence between VLSU accesses and scalar accesses into a scalar L1. The VLSU exposes a generic load/store decoupled request and response interface. This is typically converted into a coherent TileLink interface within either Rocket or Shuttle, but could be extended in the future to expose an incoherent AXI port instead.

One approach for the VLSU would be to direct all vector memory accesses into the scalar L1. While simple, such a design would induce frequent structural hazards and require a specialized host core with a specialized L1 data cache.

While the Saturn integration with Rocket does support this approach, the standard and preferred mechanism is to provide a vector-specific memory port that bypasses the L1 and accesses coherent backing memory. Figure 16 depicts such a memory system.

Saturn configurations with high DLEN would generally require higher memory bandwidth. However, scaling up the system-level interconnect to meet Saturn’s bandwidth demands may be prohibitively costly. Instead, the preferred approach for high-DLEN Saturn configs is to integrate a high-bandwidth local TCM (tightly-coupled-memory), which software should treat as a software-managed cache for vector accesses. This TCM should be tile-local and globally addressable, but not necessarily cacheable. Figure 17 depicts a Saturn configuration with a high-bandwidth TCM, but a reduced-bandwidth system interconnect.

Saturn’s integration with Shuttle supports these tile-local TCMs.

Saturn also supports integration into a system with a specialized "scatter-gather memory" (SGTCM). Unlike the standard memory interface, which supports one address per cycle for loads and one address per cycle for stores, the SGTCM interface presents an array of parallel byte-wide ports. The SGTCM is intended to be implemented as a specialized non-cacheable core-local memory.

Figure 18 depicts how the Saturn VLSU can be parameterized to bypass the block memory port to access a specialized address-generation engine for a deeply-banked scatter-gather memory. Saturn’s integration with Shuttle supports a byte-wise banked SGTCM.

Upon dispatch from the VFU into the VLSU, a vector memory instruction is written into either the load instruction queue (VLIQ) or store instruction queue (VSIQ).

Each entry in this queue contains the base offset, physical page index, and stride, as well as a bit-mask of older vector loads or stores. The VFU guarantees that all instructions dispatched into the VLSU have been cracked into operations with single-page access extents. As a result, the base offset and stride are stored as 12 bits of page offset, while a physical page index provides the upper bits of the address. Each entry additionally contains the vstart, vl, segstart, and segend settings of this instruction, along with all the fields for addressing mode, element width, index width, and mask control.

The entry also contains a bound (extent) for the instruction’s memory access within its accessed page. This is derived from the base offset, stride, vl, and addressing mode settings of the instruction, but is encoded directly within the entry to enable fast disambiguation checks. Instructions with indexed accesses are marked conservatively as accessing the entire page.

Memory disambiguation checks are performed using a CAM-like structure over all the entries in the VLIQ or VSIQ to find the entries accessing the same page. The base and extent of a given access can be checked against the base and extent of the access in the entry to determine if overlap exists. Both vector-vector and vector-scalar ordering checks use this CAM.

The address sequencer and segment buffer units derive their control signals from pointers into the inflight instruction queues. For long-chime instructions (when LMUL is high), these pointers can reference the same instruction, enabling a single instruction to occupy all the units in the load or store path. For short-chime instructions (when LMUL is low), these pointers can refer to different instructions, enabling many simultaneous inflight instructions for short-chime code in a high-latency memory system.

Saturn is responsible for stalling vector or scalar requests if an older vector or scalar request has not been made visible to the coherent memory system, and would cause a violation of the memory model if the younger request was allowed to proceed.

Section 3.3 discusses how scalar-vector memory disambiguation is managed, while this section discusses the vector-vector cases. Vector memory disambiguation is performed when instructions enter either the LAS or SAS. An instruction cannot sequence address generation operations through either address sequencer until it has cross-checked the appropriate sequencer for older instructions with overlapping access extents.

The cross-check compares the to-be-address-sequenced instruction’s extent with the extent of older instructions in the relevant inflight queue with a CAM-like structure. Since the VLIQ and VSIQ do not need to be deep, as they hold complete vector instructions, this CAM structure has tolerable cost.

Address disambiguation based on the precise extents of vector accesses, as opposed to a coarse cache-line or page-based granularity, requires a more costly CAM-like circuit. While a static-granularity address CAM can just compare the upper bits of the physical address, determine overlap by precise extents require comparators on the base and bounds of each access, where the width of each comparator is the page offset in Saturn’s case. This costly circuit is necessary to avoid stalls on false-positive conflicts between consecutive contiguous vector loads and stores.

The VLSU does not assume that the vector memory interface will respond to requests in order. This further necessitates the implementation of a load-reordering buffer (LROB). Saturn supports a LROB with as many buffer entries as possible inflight requests. Saturn additionally supports implementing the LROB with fewer buffer entries than possible inflight requests, for use in scenarios where the outer memory system generally preserves response order, but is not guaranteed to. In this configuration, the LROB will replay loads when the LROB’s buffers overflow, preserving an in-order response stream into the LMU.

The store acknowledgment unit similarly tracks the outstanding store acknowledgments. Once the final acknowledgment for a given store has arrived, that store can be freed from the VSIQ.

If integrated into a memory system that preserves load response ordering, the LROB can be omitted. Future improvements to Saturn can add this optimization.

The Address Sequencers (LAS/SAS) generate requests for all memory instructions except for indexed accesses into the SGM. The address sequencers emit requests aligned to the width of the memory interface. The sequencer can proceed with an instruction once cleared by address disambiguation.

The address sequencers effectively iterate over two nested loops. The outer loop iterates over the element index while the inner loop iterates over a "segment index" within a segment for segmented accesses. An index port and mask port provide a stream of indices/masks generated by the VU for indexed and masked operations.

Unit-strided (segmented and non-segmented) accesses do not execute the inner loop, and iterate the outer loop by the number of elements requested by the next request. These requests saturate the available memory bandwidth. Masked unit-strided loads ignore the mask settings, instead applying the mask when performing a masked write into the VRF in the VU. Masked unit-strided stores receive a byte mask from the SMU along with the data bytes, and use this to construct the store request.

Strided and indexed non-segmented accesses do not execute the inner loop, and iterate the outer loop by a single element per cycle. A mask is generated to select the active bytes within the access for the requested element. These accesses use the mask port if set by the instruction, and omit generating the request if the element is masked off.

Strided and indexed segmented accesses execute both the outer and inner loop. The inner loop iterates by the number of elements within a segment available within the next segment, while the outer loop iterates by segment index. These access the mask port if set by the instruction, and omit generating the request if the segment is masked off. Generally, these can saturate the memory bandwidth when the size of one segment is larger than the memory width.

The sequencers will stall if the memory interface is not ready or if there are no more tags to track outstanding memory accesses. When the last request for an instruction has been sent to the memory system, the pointer into the VLIQ/VSIQ for the instruction is incremented, and the next instruction to undergo address sequencing can proceed.

The merge units are general-purpose circuits that correct for misalignment of the memory system response data before the next step in the load or store paths. These can be considered a generalized form of a rotation buffer, decoupling the alignment and extent of input data from the alignment and extent of output data, and preserving latency-insensitive decoupled interfaces. Microarchitecturally, the merge unit includes two additional byte-shifters compared to a standard rotation buffer. One additional shifter enables partial dequeues of buffered data, while the other allows the buffer to "compact" many partial packets into a full response packet.

The merge units have FIFO semantics, where the enqueue into the unit specifies a base and extent of active data within the wide input vector. The merge unit rotates away the inactive bytes, compacting the active bytes into contiguous storage. The dequeue shifts the buffered data into position based on the requested base and extent. A bypass path from the enqueue to the dequeue enables full-throughput continuous dataflow for misaligned contiguous accesses.

For the LMU, the push base and extent (head and tail) are set by the address offset associated with the original memory request. For block-contiguous accesses, only the first and last beat of a single access would encode a non-aligned head or tail, respectively. For element-indexed or strided accesses where each memory request contains only a single valid element, the push head and tail denote the start and end byte of the active element. In this way, the LMU serves two roles, either rotating block-contiguous accesses or compressing indexed on strided accesses. This behavior decouples load-response alignment from writeback; regardless of addressing mode or alignment, the LMU generates aligned DLEN-wide contiguous bytes for writeback in the VU.

For segmented loads, the LMU serves an additional purpose; it enables decoupling of the load write-back scheduling performed by the datapath from the segment restructuring performed by the segment buffer. That is, the segment buffer does not necessarily proceed at DLEN bits per cycle for all segment sizes. Depending on the segment size, the segment buffer may request a sub-DLEN slice of bytes, which the LMU will provide once available.

The SMU operates as the reversed path of the LMU. The push head and tail of the SMU are usually aligned, except for the last element group when VL is misaligned. For segmented stores, the push head and tail may be set by the store segment buffer, instead of the store datapath. The pop head and tail are driven by the address alignment generated by the SAS. Notably, the SMU additionally tracks a byte-wise mask bit for masked stores, such that the mask can be applied to the generated store request.

For segmented accesses to proceed with high throughput, the LSB and SSB must "buffer" a sufficient number of responses to "transpose" a set of segments into a set of vector writebacks, or a set of vector store-data into a set of segments. Non-segmented accesses bypass the segment buffer units entirely.

Each segment buffer is implemented as a double-buffered 2D array of flops. The double-buffering enables full-rate segmented accesses. For instance, in the LSB, one half is filled by load responses while the other is drained by load writeback.

Each segment buffer is 8 rows deep to support up to 8 fields in a segment, as required by the specification. Each segment buffer is DLEN bits wide to buffer entire element groups of writeback data.

Load responses from the LMU write columns into the LSB, while the LSB emits rows into the load writeback port to the VU. Store data from the VU writes rows into the SSB, while the SSB emits columns into the SMU.

Figure 20 depicts how the LSB requests aligned segments from the LMU, stores them in a 2D segment buffer array, and presents a stream of aligned write-back data to the datapath. Notably, some configurations of NF and ELEN result in sub-optimal throughput, underutilizing the memory system. However, segmented loads and stores will always be more performant than the equivalent sequence of non-segmented loads and stores used in their place.

Some optimizations to improve the throughput of the power-of-two NF instructions have yet-to-be implemented.

Vector instructions entering the VU are immediately assigned a unique age tag and enqueued into an in-order dispatch queue (VDQ). The age tag is used to disambiguate age when determining data hazards in the out-of-order sequencing stage. This means the age tag is wide enough to assign a unique tag to every inflight operation in the backend. Age tags are recovered when instructions complete execution.

The VDQ provides a low-cost decoupling queue between the VU and the VLSU. The dequeue side of the VDQ is connected to the vector issue queues (VIQs). Each VIQ issues instructions in-order into its corresponding instruction sequencer. Execution across the VIQs may be out-of-order, as the VIQs may naturally "slip" against each other.

Each sequencing path has an independent issue queue. The load issue queue only needs to track a destination address and mask enable, while the store issue queue only needs to track a source address and mask enable. The arithmetic issue queue(s) must track up to three source operands along with the destination and mask enable. The special issue queue, used for index generation, register gathers, slides, compress, and reductions, tracks only a source operand.

There are six currently supported configuration options for the issue queue and sequencer topology.

The instruction sequencers convert a vector instruction into a sequence of operations that execute down the functional unit datapaths, one operation per cycle. The sequencers advertise the requested register file read and write addresses for the next operation, as well as the age tag for the currently sequenced instruction. If there are no structural hazards from non-pipelined functional units or register file ports and there are no data hazards against older vector instructions, a sequencer will issue an operation and update its internal state. An instruction will depart a sequencer along with the last operation it sequences, eliminating dead time between successive vector instructions.

Notably, the sequencers enact "fire-and-forget" operation issue. Once an operation is issued by a sequencer, it is guaranteed to be free of further structural or data hazards as it proceeds down the pipelined functional unit datapaths. This eliminates the need for costly operand or result queues and obviates back-pressure in the functional unit pipelines.

Due to the out-of-order execution across the different sequences, RAW, WAW and WAR hazards are all possible. Furthermore, supporting vector chaining implies that these hazards should be resolved at sub-vector-register granularity. Since Saturn is designed around DLEN-wide element groups as the base throughput of the machine, Saturn resolves data hazards at DLEN granularity. The scheduling mechanism precisely tracks which element groups an instruction or operation has yet to read-or-write to interlock the sequencers.

In Saturn, the "out-of-order instruction window" includes all instructions in the issue queues (but not the VDQ), the instructions currently in progress within the sequencers, and any operations which have not yet completed execution in the functional unit datapaths. Each instruction in this window must advertise a precise set of element groups that it will read from or write to in future cycles, along with its age tag.

The advertised information across the out-of-order window is aggregated into a pending-read and pending-write one-hot vector for each sequencer. These one-hot vectors each contain one element for each architectural element group in the VRF, which is 32xVLEN/DLEN (a typical total value being 64). These one-hot vectors are constructed using an age filter based on the age tag of the current instruction in the sequencer and the age of each operation in the out-of-order window. The age filter restricts the pending-read and pending-write vectors to only pending reads and writes from instructions older than the currently sequenced instruction.

In some cases, the relative age is unambiguous, so no age filter is needed. Instructions in the sequencer are inherently older than instructions from the feeding issue queue for that sequencer, so no age filter is needed. Sequenced operations in the functional units are inherently the oldest writes to any element group, so no age filter is needed for these either.

Each sequencer computes the element groups that will be accessed or written to by the next operation to be issued, and determines if a pending older read or write to those element groups would induce a RAW, WAR or WAR hazard. If there is no data hazard and there is no structural hazard, the operation can be issued, with the sequencer incrementing its internal element index counter, or draining the instruction.

For vector instructions with regular consecutive access patterns, the last issued operation that accesses some element group can clear the sequencer’s internal bit-mask of pending reads and/or writes. This frees younger vector instructions in other sequencers to chain off this element group as soon as possible.

The VRF is organized as a multi-ported banked array of flops. The architectural register file is striped across the banks by element group index. Neighboring element groups reside in neighboring banks. Each bank contains 3 read ports and 1 write port, to fulfill the minimum read requirements of a three-input fused-multiply-add. The generator supports generating VRFs with 1, 2, or 4 banks. Configurations that expect to keep multiple integer sequencers utilized simultaneously will prefer more banks to meet the increased read bandwidth requirements.

As an optimization, Saturn implements per-bank single-entry fall-through write buffers, effectively emulating a 3R2W memory with a 3R1W memory. Write bank conflicts between pipelined writebacks and load writebacks can result in a performance penalty. The write buffers allow two writes into a single bank, with the second write delayed onto the next cycle. Due to the bank striping of the vector register file, the delayed write is unlikely to induce further bank conflicts.

A shadow copy of vector mask register v0 is maintained in a 1R1W memory to avoid provisioning an extra read port for the bulk banked register file.

A read crossbar connects the issue port of the sequencers to the register file read ports. The read crossbar resolves structural hazards during the read stage and stalls the sequencers if necessary. The read stage also arbitrates for access to the write ports across multiple fixed-latency execution paths.

Each execution unit is composed of some set of functional units. Operations are issued to functional units along with their vector operands. Functional units can either be pipelined or iterative.

Pipelined functional units have fixed execution latencies, so once an operation is issued, it will execute without stalling. The sequencing mechanism checks for future write port conflicts on the target VRF bank across inflight and simultaneously sequenced operations to ensure that the next sequenced operation will not induce a structural hazard on the write port in the future. If a conflict is detected, the younger operation will be stalled and will likely start executing the very next cycle.

Iterative functional units have variable execution latencies or contain expensive hardware such that it is desirable to execute at a rate of just one element per cycle. Once an iterative functional unit has completed its operation on a given element, it will arbitrate for the target VRF write port, write the result, then assert readiness to accept a new operation from the sequencer.

Leveraging vector chaining is critical for achieving high utilization of the functional units. Saturn implements vector chaining at DLEN, or element-group granularity, through the vector register file (direct functional unit to functional unit bypasses are not implemented). Chaining enables a dependent vector instruction to begin sequencing operations as soon as its parent instruction writes back its first element group, assuming no other structural or data hazards.

Chaining only occurs between instructions occupying different sequencers, as the sequencers are single-issue in-order. All sequencers can chain between each other. For example, arithmetic operations may chain off of memory operations and vice-versa. When using a configuration with multiple execute sequencers, independent arithmetic operations may chain off each other and overlap execution. Optimized vector code should seek to load-balance instructions across the sequencers such that chaining might occur.

To maximize the utilization of all functional units, the issue structure of the machine should be considered. The issue queue structure and depths are parameterized in Saturn and may differ across different configurations.

In the "Unified" structure, all vector arithmetic instructions, whether integer or floating-point, execute from the same sequencer, so no parallel arithmetic execution is possible. In the "Shared" and "Split" structures, independent sequencers drive the integer and floating-point execution units, so parallel execution is possible under certain conditions. In the "Shared" configuration, since the sequencers share a single in-order issue queue, interleaving vector instructions across the integer and floating-point paths is critical. In the "Split" configuration, each sequencer has its own issue queue, reducing the importance of software-scheduled load balancing.

When the issue queue sizes are shallow, the issue queues may fill up and back-pressure the decoupling queue, and consequently, the scalar core’s frontend. Even modest configurations of Saturn should contain enough buffering within the issue and decoupling queues to avoid back-pressure for well-load-balanced code.

Saturn is designed to be performant even when integrated with a compact, single-issue, in-order scalar core. Specific details related to Rocket and Shuttle can be found in Chapter 3. When writing loops, scalar bookkeeping instructions should be scheduled to overlap with the execution of the vector instructions.

Saturn configurations with short chime lengths are preferably integrated with Shuttle, as short-chime instructions generally require higher frontend instruction throughput. When operating on long vector lengths with long chimes, the performance difference between Rocket and Shuttle diminishes. As discussed in Chapter 3, the handling of vset instructions varies across different scalar cores. This may induce pipeline bubbles and should be considered when writing kernels.

Due to the post-commit execution of vector instructions, vector instructions that write scalar registers should be used sparingly. As the supported scalar cores do not implement speculative execution, RAW dependencies on vector-produced scalar results will cause the scalar pipeline to stall. These should be avoided in inner loops.

A single vector instruction requires VLEN/DLEN (the chime length) cycles of occupancy on most DLEN-wide functional unit datapaths. When this chime length is less than the pipeline depth of the relevant functional unit, the proceeding instruction will stall due to a data dependency on the yet-to-be-produced writeback data. This situation is rare due to the support for chaining, but might still appear in a sequence of low-LMUL floating point instructions.

To saturate the FMA units in this scenario, either a longer LMUL should be used, or independent FMAs must be scheduled back-to-back. Generally, performant code should use the highest LMUL possible that avoids vector register spilling.

Refer to Chapter 5 for details on each of the vector functional units and their default pipeline depths.

Saturn’s implementation strives to implement complex instructions efficiently. In particular, the segmented loads and stores that reformat memory into vector registers are implemented to be able to fully utilize the wide memory port. As a general rule, using such complex instructions should never induce a performance penalty over an equivalent longer sequence of simpler instructions. Thus, programmers should use complex instructions whenever possible, even if the kernel could be equivalently implemented by a longer sequence of simpler instructions.

Currently, Saturn cannot maintain full utilization for element-wise reductions due to an insufficient number of accumulator registers to track intermediate values. Classic techniques for vectorizing reduction operations should be utilized where possible. For example, applicable reduction loops should be expressed as a sequence of element-wise operations before a final reduction across all elements in the tail code.

Saturn draws from a long line of prior vector units.

The Saturn project began in the fall of 2021 with a vision to develop a standard base vector implementation for hosting specialized accelerators. Early iterations of Saturn explored different design approaches to compact vector implementations, including shallow-pipeline systems, merged scalar-vector implementations, and partial implementations of the full RISC-V Vector ISA.

Saturn 0.1 implemented an aggressive ahead-of-commit vector pipeline, with a custom scalar core implementation. Saturn 0.1 had unstable and unpredictable performance characteristics due to the complex set of interlocks needed to enable parallel vector and scalar execution. Specifically, allowing for variable-chime instruction execution significantly increased the complexity of the tightly integrated vector unit due to the implications on instruction occupancies of shared scalar/vector pipeline structures. Only a small subset of vector instructions were supported on this implementation.

Saturn 0.2 implemented a post-commit vector unit behind a custom-designed superscalar core with a very shallow pipeline and no support for virtual memory. A shared scalar-vector load-store-unit executed both scalar and vector memory operations post-commit, with a specialized high-bandwidth non-blocking L1 data cache fulfilling both scalar and vector memory requests. A very shallow three-stage pipeline pre-commit attempted to mitigate high scalar-load-use penalties, but this precluded support for virtual memory. Saturn 0.2 did implement the first version of the instruction sequencing mechanism used in Saturn 1.0.

Saturn 0.9 began development in the fall of 2023 with a complete rewrite over prior versions. The desire to support complete RVV alongside a fully-featured scalar core was motivated by the observation that accelerator architects generally expect mature software stacks and well-behaved core implementations for scalar control code. Furthermore, learnings from Saturn 0.1 and 0.2 suggested that a "three-stage" load path, with ahead-of-commit fault checking, early post-commit load-address-generation, and late post-commit load write-back, would adequately address performance requirements while remaining compliant to the ISA.

The decoupled-access-execute architecture with an aggressive yet precise vector frontend became the dominant design paradigm driving the rest of Saturn 0.9’s microarchitecture. The implementation of Saturn focused on developing a robust precise fault mechanism, load-store path, and dynamic out-of-order instruction issue in that order. Once the microarchitecture around dynamic vector scheduling with chaining was refined, implementation of the functional units proceeded expediently. After the first vector integer ALU was implemented, the rest of the vector instructions were brought up in three months.

At the time of writing, Saturn 0.9 has been taped out twice. In spring 2024, a conservative multi-core Saturn configuration attached to Rocket was taped out out as part of Berkeley’s undergraduate tapeout course. Concurrently, a more aggressive heterogeneous clustered multi-core of Saturns attached to Shuttle cores was taped out in a research SoC.

Saturn 1.0 was developed alongside Shuttle 2.0, leveraging physical design experience from the prior tapeouts. Saturn 1.0 contained many critical path and area optimizations and was released in the fall of 2024 along with this microarchitecture manual.

Jerry Zhao led the Saturn project and implemented the instruction scheduling mechanism, load-store unit, and scalar-core integration. Daniel Grubb led the implementation of the SIMD floating-point units and the Saturn physical implementation and contributed to the core subsystems of the vector backend. Miles Rusch implemented several variants of SIMD integer multipliers. Tianrui Wei assisted with early versions of Saturn and performed key work that would evolve into the 1.0 version.

Kevin He, Nico Casteneda, and Mihai Tudor led a tapeout of a conservative Saturn with Rocket implementation as part of Berkeley’s undergraduate tapeout course. Kevin Anderson, Daniel Grubb, and Vikram Jain led a research tapeout of a DSP-optimized chip with heterogeneous clusters of Saturns.

Many people have contributed to projects that have led to Saturn. John Fang developed LEM, exploring the challenges of generating efficient vector decode structures. Albert Ou’s thesis work highlighted use cases for Saturn-like vector units in accelerating DSP kernels.

Lastly, the wisdom of former graduate students who have developed vector units during their tenures has proven instrumental to the development of Saturn. Conversations with Albert Ou, Colin Schmidt, Andrew Waterman and Chris Batten have all been insightful.

Research was partially funded by SLICE Lab industrial sponsors and affiliates, and by the NSF CCRI ENS Chipyard Award #2016662. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.