su2hmc
Loading...
Searching...
No Matches
sizes.h File Reference

Defines the constants of the code and other parameters for loop dimensions. Each subroutine includes these definitions using: More...

#include <cuda.h>
#include <cuda_runtime_api.h>
#include <cublas_v2.h>
#include <complex.h>
Include dependency graph for sizes.h:
This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Macros

#define cudaDeviceSynchronise()
 Get rid of that bastardised yankee English.
#define Complex_f   float complex
 Single precision complex number.
#define Complex   double complex
 Double precision complex number.
#define FILELEN   64
 Default file name length.
#define nx   24
 Lattice x extent.
#define ny   nx
 Lattice y extent. We normally use cubic lattices so this is the same as nx.
#define nz   nx
 Lattice z extent. We normally use cubic lattices so this is the same as nx.
#define nt   24
 Lattice temporal extent. This also corresponds to the inverse temperature.
#define gvol   (nx*ny*nz*nt)
 Lattice volume.
#define gvol3   (nx*ny*nz)
 Lattice spatial volume.
#define npx   1
 Processor grid x extent. This must be a divisor of nx.
#define npy   npx
 Processor grid y extent.
#define npz   npx
 Processor grid z extent.
#define npt   1
 Processor grid t extent.
#define nproc   (npx*npy*npz*npt)
 Number of processors for MPI.
#define nthreads   8
 Number of threads for OpenMP, which can be overwritten at runtime.
#define ksizex   (nx/npx)
 Sublattice x extent.
#define ksizey   (ny/npy)
 Sublattice y extent.
#define ksizez   (nz/npz)
 Sublattice z extent.
#define ksize   ksizex
 Sublattice spatial extent for a cubic lattice.
#define ksizet   (nt/npt)
 Sublattice t extent.
#define nf   1
 Fermion flavours (double it).
#define kvol   (ksizet*ksizez*ksizey*ksizex)
 Sublattice volume.
#define kvol3   (ksizez*ksizey*ksizex)
 Sublattice spatial volume.
#define niterc   gvol
 Hard limit for runaway trajectories in Conjugate gradient.
#define nc   2
 Colours.
#define nadj   3
 adjacent spatial indices
#define ndirac   4
 Dirac indices.
#define ndim   4
 Dimensions.
#define ngorkov   8
 Gor'kov indices.
#define kmom   (ndim*nadj*kvol)
 sublattice momentum sites
#define kferm   (nc*ngorkov*kvol)
 sublattice size including Gor'kov indices
#define kferm2   (nc*ndirac*kvol)
 sublattice size including Dirac indices
#define halox   0
 x Halo size
#define haloy   0
 y Halo size
#define haloz   0
 z Halo size
#define halot   0
 t Halo size
#define halo   (2*(halox+haloy+haloz+halot))
 Total Halo size.
#define kvolHalo   (kvol+halo)
 Subvolume + halo size.
#define kfermHalo   (nc*ngorkov*kvolHalo)
 Gor'kov lattice and halo.
#define kferm2Halo   (nc*ndirac*kvolHalo)
 Dirac lattice and halo.
#define kmomHalo   (ndim*nadj*kvolHalo)
 Momentum lattice and halo.
#define respbp   3.2E-5
 Conjugate gradient residue for \(\langle\bar{\Psi}\Psi\rangle\).
#define rescgg   2.26E-5
 Conjugate gradient residue for update.
#define rescga   2.26E-8
 Conjugate gradient residue for acceptance.
#define AVX   16
 Alignment of arrays. 64 for AVX-512, 32 for AVX/AVX2. 16 for SSE. Since AVX is standard on modern x86 machines I've called it that.
#define USE_BLAS
 Flag to choose BLAS path.

Variables

cublasHandle_t cublas_handle
 Handle for cuBLAS.
cublasStatus_t cublas_status
 Status of cuBLAS for error reporting.
cudaMemPool_t mempool
 Memory pool for Async allocations.
dim3 dimBlock
 Default block size. Usually 128.
dim3 dimGrid
 Default grid size. First component is normally nt. Second and third depend whatever is needed to get the block size to 128.
dim3 dimBlockOne
 block size of one
dim3 dimGridOne
 Grid size of one.

Detailed Description

Defines the constants of the code and other parameters for loop dimensions. Each subroutine includes these definitions using:

INCLUDE sizes.h  

Or at least they used to. But this is the C version, so no need for that. Include once in each source file and it should work just fine.

The only potential snag is that including the header makes its content global in scope. Where this could cause an issue we shall instead declare the variable inside the function and its scope shall override that of the global variable.

I've tried keeping the comments as close to the original as possible. Of course some magic numbers will be redundant and some will need to be added or redefined. Anything that was commented out has been left commented out with a second C commented out version beneath it.

Author
D. Lawlor September 2020

Definition in file sizes.h.

Macro Definition Documentation

◆ AVX

#define AVX   16

Alignment of arrays. 64 for AVX-512, 32 for AVX/AVX2. 16 for SSE. Since AVX is standard on modern x86 machines I've called it that.

Definition at line 279 of file sizes.h.

◆ Complex

#define Complex   double complex

Double precision complex number.

Definition at line 64 of file sizes.h.

◆ Complex_f

#define Complex_f   float complex

Single precision complex number.

Definition at line 62 of file sizes.h.

◆ cudaDeviceSynchronise

#define cudaDeviceSynchronise ( )
Value:
cudaDeviceSynchronize()

Get rid of that bastardised yankee English.

Definition at line 53 of file sizes.h.

◆ FILELEN

#define FILELEN   64

Default file name length.

Definition at line 68 of file sizes.h.

◆ gvol

#define gvol   (nx*ny*nz*nt)

Lattice volume.

Definition at line 98 of file sizes.h.

◆ gvol3

#define gvol3   (nx*ny*nz)

Lattice spatial volume.

Definition at line 100 of file sizes.h.

◆ halo

#define halo   (2*(halox+haloy+haloz+halot))

Total Halo size.

Definition at line 231 of file sizes.h.

◆ halot

#define halot   0

t Halo size

Definition at line 228 of file sizes.h.

◆ halox

#define halox   0

x Halo size

MPI Halos

For those who may not have used MPI Before, halos are just a bit of padding we put outside of the sublattices we're using in MPI so we can look at terms outside the sublattice we're actively working on with that process.

Definition at line 210 of file sizes.h.

◆ haloy

#define haloy   0

y Halo size

Definition at line 216 of file sizes.h.

◆ haloz

#define haloz   0

z Halo size

Definition at line 222 of file sizes.h.

◆ kferm

#define kferm   (nc*ngorkov*kvol)

sublattice size including Gor'kov indices

Definition at line 195 of file sizes.h.

◆ kferm2

#define kferm2   (nc*ndirac*kvol)

sublattice size including Dirac indices

Definition at line 197 of file sizes.h.

◆ kferm2Halo

#define kferm2Halo   (nc*ndirac*kvolHalo)

Dirac lattice and halo.

Definition at line 238 of file sizes.h.

◆ kfermHalo

#define kfermHalo   (nc*ngorkov*kvolHalo)

Gor'kov lattice and halo.

Definition at line 236 of file sizes.h.

◆ kmom

#define kmom   (ndim*nadj*kvol)

sublattice momentum sites

Definition at line 193 of file sizes.h.

◆ kmomHalo

#define kmomHalo   (ndim*nadj*kvolHalo)

Momentum lattice and halo.

Definition at line 240 of file sizes.h.

◆ ksize

#define ksize   ksizex

Sublattice spatial extent for a cubic lattice.

Definition at line 155 of file sizes.h.

◆ ksizet

#define ksizet   (nt/npt)

Sublattice t extent.

Definition at line 158 of file sizes.h.

◆ ksizex

#define ksizex   (nx/npx)

Sublattice x extent.

Definition at line 148 of file sizes.h.

◆ ksizey

#define ksizey   (ny/npy)

Sublattice y extent.

Definition at line 150 of file sizes.h.

◆ ksizez

#define ksizez   (nz/npz)

Sublattice z extent.

Definition at line 152 of file sizes.h.

◆ kvol

#define kvol   (ksizet*ksizez*ksizey*ksizex)

Sublattice volume.

Definition at line 163 of file sizes.h.

◆ kvol3

#define kvol3   (ksizez*ksizey*ksizex)

Sublattice spatial volume.

Definition at line 165 of file sizes.h.

◆ kvolHalo

#define kvolHalo   (kvol+halo)

Subvolume + halo size.

Definition at line 234 of file sizes.h.

◆ nadj

#define nadj   3

adjacent spatial indices

Definition at line 184 of file sizes.h.

◆ nc

#define nc   2

Colours.

Definition at line 182 of file sizes.h.

◆ ndim

#define ndim   4

Dimensions.

Definition at line 188 of file sizes.h.

◆ ndirac

#define ndirac   4

Dirac indices.

Definition at line 186 of file sizes.h.

◆ nf

#define nf   1

Fermion flavours (double it).

Definition at line 160 of file sizes.h.

◆ ngorkov

#define ngorkov   8

Gor'kov indices.

Definition at line 190 of file sizes.h.

◆ niterc

#define niterc   gvol

Hard limit for runaway trajectories in Conjugate gradient.

Definition at line 172 of file sizes.h.

◆ nproc

#define nproc   (npx*npy*npz*npt)

Number of processors for MPI.

Definition at line 138 of file sizes.h.

◆ npt

#define npt   1

Processor grid t extent.

Definition at line 130 of file sizes.h.

◆ npx

#define npx   1

Processor grid x extent. This must be a divisor of nx.

Definition at line 103 of file sizes.h.

◆ npy

#define npy   npx

Processor grid y extent.

Definition at line 114 of file sizes.h.

◆ npz

#define npz   npx

Processor grid z extent.

Definition at line 122 of file sizes.h.

◆ nt

#define nt   24

Lattice temporal extent. This also corresponds to the inverse temperature.

Definition at line 92 of file sizes.h.

◆ nthreads

#define nthreads   8

Number of threads for OpenMP, which can be overwritten at runtime.

Definition at line 144 of file sizes.h.

◆ nx

#define nx   24

Lattice x extent.

Definition at line 72 of file sizes.h.

◆ ny

#define ny   nx

Lattice y extent. We normally use cubic lattices so this is the same as nx.

Definition at line 80 of file sizes.h.

◆ nz

#define nz   nx

Lattice z extent. We normally use cubic lattices so this is the same as nx.

Definition at line 86 of file sizes.h.

◆ rescga

#define rescga   2.26E-8

Conjugate gradient residue for acceptance.

Definition at line 253 of file sizes.h.

◆ rescgg

#define rescgg   2.26E-5

Conjugate gradient residue for update.

Definition at line 251 of file sizes.h.

◆ respbp

#define respbp   3.2E-5

Conjugate gradient residue for \(\langle\bar{\Psi}\Psi\rangle\).

Definition at line 249 of file sizes.h.

◆ USE_BLAS

#define USE_BLAS

Flag to choose BLAS path.

Definition at line 303 of file sizes.h.

Variable Documentation

◆ cublas_handle

cublasHandle_t cublas_handle
extern

Handle for cuBLAS.

Definition at line 47 of file main.c.

◆ cublas_status

cublasStatus_t cublas_status
extern

Status of cuBLAS for error reporting.

Definition at line 48 of file main.c.

◆ dimBlock

dim3 dimBlock
extern

Default block size. Usually 128.

Grids and Blocks

Threads are grouped together to form warps of 32 threads best to keep the block dimension (ksizex*ksizey) multiples of 32, usually between 128 and 256 Note that from Volta/Turing each SM (group of processors) is smaller than on previous and subsequent generations of GPUs. This means less threads accessing the same number of registers which can increase occupancy for register heavy kernels.

Definition at line 25 of file cusu2hmc.cu.

◆ dimBlockOne

dim3 dimBlockOne
extern

block size of one

Definition at line 22 of file cusu2hmc.cu.

◆ dimGrid

dim3 dimGrid
extern

Default grid size. First component is normally nt. Second and third depend whatever is needed to get the block size to 128.

Definition at line 27 of file cusu2hmc.cu.

◆ dimGridOne

dim3 dimGridOne
extern

Grid size of one.

Definition at line 23 of file cusu2hmc.cu.

◆ mempool

cudaMemPool_t mempool
extern

Memory pool for Async allocations.

Definition at line 49 of file main.c.