PAPI on Theta

Help Desk

Theta

References

Introduction

The Performance Application Programming Interface (PAPI) provides a standardized user level API for accessing processor and other system programmer counter information (such as instruction counts, cache misses, etc.).

PAPI details are provided on the PAPI Website.

PAPI on Theta

PAPI is available on Theta through the “papi” or “perftools-base” modules.

The preset and native events available on Theta may be listed by running the 'papi_avail' and 'papi_native_avail' utilities on a compute node via the qsub command. Accessing native events requires using the event names reported by 'papi_native_avail' as a string and converting them into an event code with the function PAPI_event_name_to_code().

PAPI via papi module on Theta

jkwack@thetalogin5:~> qsub -I -n 1 -t 60 -q debug-cache-quad -A Performance

Connecting to thetamom3 for interactive qsub...

Job routed to queue "debug-cache-quad".

Memory mode set to cache quad for queue debug-cache-quad

Wait for job 414557 to start...

Opening interactive session to 3834

jkwack@thetamom3:~> module load papi

jkwack@thetamom3:~> module show papi

-------------------------------------------------------------------

/opt/cray/pe/modulefiles/papi/5.7.0.2:


conflict papi

conflict perftools

conflict perftools-lite

conflict perftools-lite-events

conflict perftools-lite-gpu

conflict perftools-lite-hbm

conflict perftools-lite-loops

conflict perftools-nwpc

conflict perftools-preload

conflict perftools-base

prepend-path PE_PKGCONFIG_LIBS papi:pfm

prepend-path PKG_CONFIG_PATH /opt/cray/pe/papi/5.7.0.2/lib64/pkgconfig

prepend-path PE_PKGCONFIG_PRODUCTS PE_PAPI

setenv PE_PAPI_PKGCONFIG_VARIABLES PE_PAPI_ACCEL_LIBS_@accelerator@ PE_PAPI_ACCEL_FAMILY_LIBS_@accelerator_family@

setenv PE_PAPI_ACCEL_LIBS_nvidia35 ,-lcupti,-lcudart,-lcuda

setenv PE_PAPI_ACCEL_LIBS

setenv PE_PAPI_ACCEL_FAMILY_LIBS_nvidia ,-lcupti,-lcudart,-lcuda

setenv PE_PAPI_ACCELL_FAMILY_LIBS

prepend-path CRAY_LD_LIBRARY_PATH /opt/cray/pe/papi/5.7.0.2/lib64

prepend-path PATH /opt/cray/pe/papi/5.7.0.2/bin

prepend-path MANPATH /opt/cray/pe/papi/5.7.0.2/share/man

setenv PAPI_VERSION 5.7.0.2

setenv LIBPFM_DISABLED_PMUS rapl,fam15h_nb

module-whatis PAPI - The Performance API (PAPI) project specifies a standard application programming interface (API) for accessing hardware performance counters available on most Cray systems.

-------------------------------------------------------------------


jkwack@thetamom3:~> papi_avail

Available PAPI preset and user defined events plus hardware information.

--------------------------------------------------------------------------------

PAPI version             : 5.7.0.2

Operating system         : Linux 4.4.103-6.38_4.0.154-cray_ari_s

Vendor string and code   : GenuineIntel (1, 0x1)

Model string and code    : Intel(R) Xeon(R) CPU E5-2695 v4 @ 2.10GHz (79, 0x4f)

CPU revision             : 1.000000

CPUID                    : Family/Model/Stepping 6/79/1, 0x06/0x4f/0x01

CPU Max MHz              : 2101

CPU Min MHz              : 1200

Total cores              : 72

SMT threads per core     : 2

Cores per socket         : 18

Sockets                  : 2

Cores per NUMA region    : 36

NUMA regions             : 2

Running in a VM          : no

Number Hardware Counters : 11

Max Multiplex Counters   : 384

Fast counter read (rdpmc): no

--------------------------------------------------------------------------------


================================================================================

  PAPI Preset Events

================================================================================

    Name        Code    Avail Deriv Description (Note)

PAPI_L1_DCM  0x80000000  Yes   No   Level 1 data cache misses

PAPI_L1_ICM  0x80000001  Yes   No   Level 1 instruction cache misses

PAPI_L2_DCM  0x80000002  Yes   Yes  Level 2 data cache misses

PAPI_L2_ICM  0x80000003  Yes   No   Level 2 instruction cache misses

PAPI_L3_DCM  0x80000004  No    No   Level 3 data cache misses

PAPI_L3_ICM  0x80000005  No    No   Level 3 instruction cache misses

PAPI_L1_TCM  0x80000006  Yes   Yes  Level 1 cache misses

PAPI_L2_TCM  0x80000007  Yes   No   Level 2 cache misses

PAPI_L3_TCM  0x80000008  Yes   No   Level 3 cache misses

PAPI_CA_SNP  0x80000009  Yes   No   Requests for a snoop

PAPI_CA_SHR  0x8000000a  Yes   No   Requests for exclusive access to shared cache line

PAPI_CA_CLN  0x8000000b  Yes   No   Requests for exclusive access to clean cache line

PAPI_CA_INV  0x8000000c  Yes   No   Requests for cache line invalidation

PAPI_CA_ITV  0x8000000d  Yes   No   Requests for cache line intervention

PAPI_L3_LDM  0x8000000e  Yes   No   Level 3 load misses

PAPI_L3_STM  0x8000000f  No    No   Level 3 store misses

PAPI_BRU_IDL 0x80000010  No    No   Cycles branch units are idle

PAPI_FXU_IDL 0x80000011  No    No   Cycles integer units are idle

PAPI_FPU_IDL 0x80000012  No    No   Cycles floating point units are idle

PAPI_LSU_IDL 0x80000013  No    No   Cycles load/store units are idle

PAPI_TLB_DM  0x80000014  Yes   Yes  Data translation lookaside buffer misses

PAPI_TLB_IM  0x80000015  Yes   No   Instruction translation lookaside buffer misses

PAPI_TLB_TL  0x80000016  No    No   Total translation lookaside buffer misses

PAPI_L1_LDM  0x80000017  Yes   No   Level 1 load misses

PAPI_L1_STM  0x80000018  Yes   No   Level 1 store misses

PAPI_L2_LDM  0x80000019  Yes   No   Level 2 load misses

PAPI_L2_STM  0x8000001a  Yes   No   Level 2 store misses

PAPI_BTAC_M  0x8000001b  No    No   Branch target address cache misses

PAPI_PRF_DM  0x8000001c  Yes   No   Data prefetch cache misses

PAPI_L3_DCH  0x8000001d  No    No   Level 3 data cache hits

PAPI_TLB_SD  0x8000001e  No    No   Translation lookaside buffer shootdowns

PAPI_CSR_FAL 0x8000001f  No    No   Failed store conditional instructions

PAPI_CSR_SUC 0x80000020  No    No   Successful store conditional instructions

PAPI_CSR_TOT 0x80000021  No    No   Total store conditional instructions

PAPI_MEM_SCY 0x80000022  No    No   Cycles Stalled Waiting for memory accesses

PAPI_MEM_RCY 0x80000023  No    No   Cycles Stalled Waiting for memory Reads

PAPI_MEM_WCY 0x80000024  Yes   No   Cycles Stalled Waiting for memory writes

PAPI_STL_ICY 0x80000025  Yes   No   Cycles with no instruction issue

PAPI_FUL_ICY 0x80000026  Yes   Yes  Cycles with maximum instruction issue

PAPI_STL_CCY 0x80000027  Yes   No   Cycles with no instructions completed

PAPI_FUL_CCY 0x80000028  Yes   No   Cycles with maximum instructions completed

PAPI_HW_INT  0x80000029  No    No   Hardware interrupts

PAPI_BR_UCN  0x8000002a  Yes   Yes  Unconditional branch instructions

PAPI_BR_CN   0x8000002b  Yes   No   Conditional branch instructions

PAPI_BR_TKN  0x8000002c  Yes   Yes  Conditional branch instructions taken

PAPI_BR_NTK  0x8000002d  Yes   No   Conditional branch instructions not taken

PAPI_BR_MSP  0x8000002e  Yes   No   Conditional branch instructions mispredicted

PAPI_BR_PRC  0x8000002f  Yes   Yes  Conditional branch instructions correctly predicted

PAPI_FMA_INS 0x80000030  No    No   FMA instructions completed

PAPI_TOT_IIS 0x80000031  No    No   Instructions issued

PAPI_TOT_INS 0x80000032  Yes   No   Instructions completed

PAPI_INT_INS 0x80000033  No    No   Integer instructions

PAPI_FP_INS  0x80000034  No    No   Floating point instructions

PAPI_LD_INS  0x80000035  Yes   No   Load instructions

PAPI_SR_INS  0x80000036  Yes   No   Store instructions

PAPI_BR_INS  0x80000037  Yes   No   Branch instructions

PAPI_VEC_INS 0x80000038  No    No   Vector/SIMD instructions (could include integer)

PAPI_RES_STL 0x80000039  Yes   No   Cycles stalled on any resource

PAPI_FP_STAL 0x8000003a  No    No   Cycles the FP unit(s) are stalled

PAPI_TOT_CYC 0x8000003b  Yes   No   Total cycles

PAPI_LST_INS 0x8000003c  Yes   Yes  Load/store instructions completed

PAPI_SYC_INS 0x8000003d  No    No   Synchronization instructions completed

PAPI_L1_DCH  0x8000003e  No    No   Level 1 data cache hits

PAPI_L2_DCH  0x8000003f  No    No   Level 2 data cache hits

PAPI_L1_DCA  0x80000040  No    No   Level 1 data cache accesses

PAPI_L2_DCA  0x80000041  Yes   No   Level 2 data cache accesses

PAPI_L3_DCA  0x80000042  Yes   Yes  Level 3 data cache accesses

PAPI_L1_DCR  0x80000043  No    No   Level 1 data cache reads

PAPI_L2_DCR  0x80000044  Yes   No   Level 2 data cache reads

PAPI_L3_DCR  0x80000045  Yes   No   Level 3 data cache reads

PAPI_L1_DCW  0x80000046  No    No   Level 1 data cache writes

PAPI_L2_DCW  0x80000047  Yes   No   Level 2 data cache writes

PAPI_L3_DCW  0x80000048  Yes   No   Level 3 data cache writes

PAPI_L1_ICH  0x80000049  No    No   Level 1 instruction cache hits

PAPI_L2_ICH  0x8000004a  Yes   No   Level 2 instruction cache hits

PAPI_L3_ICH  0x8000004b  No    No   Level 3 instruction cache hits

PAPI_L1_ICA  0x8000004c  No    No   Level 1 instruction cache accesses

PAPI_L2_ICA  0x8000004d  Yes   No   Level 2 instruction cache accesses

PAPI_L3_ICA  0x8000004e  Yes   No   Level 3 instruction cache accesses

PAPI_L1_ICR  0x8000004f  No    No   Level 1 instruction cache reads

PAPI_L2_ICR  0x80000050  Yes   No   Level 2 instruction cache reads

PAPI_L3_ICR  0x80000051  Yes   No   Level 3 instruction cache reads

PAPI_L1_ICW  0x80000052  No    No   Level 1 instruction cache writes

PAPI_L2_ICW  0x80000053  No    No   Level 2 instruction cache writes

PAPI_L3_ICW  0x80000054  No    No   Level 3 instruction cache writes

PAPI_L1_TCH  0x80000055  No    No   Level 1 total cache hits

PAPI_L2_TCH  0x80000056  No    No   Level 2 total cache hits

PAPI_L3_TCH  0x80000057  No    No   Level 3 total cache hits

PAPI_L1_TCA  0x80000058  No    No   Level 1 total cache accesses

PAPI_L2_TCA  0x80000059  Yes   Yes  Level 2 total cache accesses

PAPI_L3_TCA  0x8000005a  Yes   No   Level 3 total cache accesses

PAPI_L1_TCR  0x8000005b  No    No   Level 1 total cache reads

PAPI_L2_TCR  0x8000005c  Yes   Yes  Level 2 total cache reads

PAPI_L3_TCR  0x8000005d  Yes   Yes  Level 3 total cache reads

PAPI_L1_TCW  0x8000005e  No    No   Level 1 total cache writes

PAPI_L2_TCW  0x8000005f  Yes   No   Level 2 total cache writes

PAPI_L3_TCW  0x80000060  Yes   No   Level 3 total cache writes

PAPI_FML_INS 0x80000061  No    No   Floating point multiply instructions

PAPI_FAD_INS 0x80000062  No    No   Floating point add instructions

PAPI_FDV_INS 0x80000063  No    No   Floating point divide instructions

PAPI_FSQ_INS 0x80000064  No    No   Floating point square root instructions

PAPI_FNV_INS 0x80000065  No    No   Floating point inverse instructions

PAPI_FP_OPS  0x80000066  No    No   Floating point operations

PAPI_SP_OPS  0x80000067  Yes   Yes  Floating point operations; optimized to count scaled single precision vector operations

PAPI_DP_OPS  0x80000068  Yes   Yes  Floating point operations; optimized to count scaled double precision vector operations

PAPI_VEC_SP  0x80000069  Yes   Yes  Single precision vector/SIMD instructions

PAPI_VEC_DP  0x8000006a  Yes   Yes  Double precision vector/SIMD instructions

PAPI_REF_CYC 0x8000006b  Yes   No   Reference clock cycles

--------------------------------------------------------------------------------

Of 108 possible events, 60 are available, of which 16 are derived.



jkwack@thetamom3:~> which papi_avail

/opt/cray/pe/papi/5.7.0.2/bin/papi_avail

PAPI via perftools-base module on Theta

jkwack@thetamom3:~> module unload papi

jkwack@thetamom3:~> module load perftools-base

jkwack@thetamom3:~> module show perftools-base

-------------------------------------------------------------------

/opt/cray/pe/modulefiles/perftools-base/7.0.5:


setenv PERFTOOLS_VERSION 7.0.5

module-whatis The Performance Tools module sets up environments for CrayPat, Apprentice2 and Reveal

conflict perftools-base

conflict perftools

conflict perftools-nwpc

conflict perftools-lite

conflict perftools-lite-loops

conflict perftools-lite-events

conflict perftools-lite-gpu

conflict perftools-lite-hbm

conflict perftools-preload

conflict xt-mpich2

conflict papi

conflict craypat

conflict cuda

conflict cudatools

module use /opt/cray/pe/perftools/7.0.5/modulefiles

prepend-path MODULEPATH /opt/cray/pe/perftools/7.0.5/modulefiles

setenv CRAYPAT_ALPS_COMPONENT /opt/cray/pe/perftools/7.0.5/sbin/pat_alps

setenv PAT_BUILD_PAPI_BASEDIR /opt/cray/pe/papi/5.6.0.5

prepend-path MANPATH /opt/cray/pe/papi/5.6.0.5/share/pdoc/man

prepend-path PATH /opt/cray/pe/papi/5.6.0.5/bin

prepend-path LD_LIBRARY_PATH /opt/cray/pe/papi/5.6.0.5/lib64

setenv CHPL_CG_CPP_LINES 1

setenv PAT_REPORT_PRUNE_NAME _cray$mt_execute_,_cray$mt_start_,__cray_hwpc_,f_cray_hwpc_,cstart,__pat_,pat_region_,PAT_,OMP.slave_loop,slave_entry,_new_slave_entry,_thread_pool_slave_entry,THREAD_POOL_join,__libc_start_main,_start,__start,start_thread,__wrap_,UPC_ADIO_,_upc_,upc_,__caf_,__pgas_,syscall,__device_stub

setenv OFFLOAD_INIT on_start

setenv CRAYPAT_LD_LIBRARY_PATH /opt/cray/pe/gcc-libs:/opt/cray/gcc-libs:/opt/cray/pe/perftools/7.0.5/lib64

prepend-path PATH /opt/cray/pe/perftools/7.0.5/bin

prepend-path MANPATH /opt/cray/pe/perftools/7.0.5/man

setenv CRAYPAT_OPTS_EXECUTABLE sbin/pat-opts

setenv CRAYPAT_ROOT /opt/cray/pe/perftools/7.0.5

setenv APP2_STATE 7.0.5

prepend-path CRAY_LD_LIBRARY_PATH /opt/cray/pe/perftools/7.0.5/lib64

append-path PE_PRODUCT_LIST PERFTOOLS

append-path PE_PRODUCT_LIST CRAYPAT

-------------------------------------------------------------------


jkwack@thetamom3:~> papi_avail

Available PAPI preset and user defined events plus hardware information.

--------------------------------------------------------------------------------

PAPI version             : 5.6.0.5

Operating system         : Linux 4.4.103-6.38_4.0.154-cray_ari_s

Vendor string and code   : GenuineIntel (1, 0x1)

Model string and code    : Intel(R) Xeon(R) CPU E5-2695 v4 @ 2.10GHz (79, 0x4f)

CPU revision             : 1.000000

CPUID                    : Family/Model/Stepping 6/79/1, 0x06/0x4f/0x01

CPU Max MHz              : 2101

CPU Min MHz              : 1200

Total cores              : 72

SMT threads per core     : 2

Cores per socket         : 18

Sockets                  : 2

Cores per NUMA region    : 36

NUMA regions             : 2

Running in a VM          : no

Number Hardware Counters : 11

Max Multiplex Counters   : 384

Fast counter read (rdpmc): no

--------------------------------------------------------------------------------


================================================================================

  PAPI Preset Events

================================================================================

    Name        Code    Avail Deriv Description (Note)

PAPI_L1_DCM  0x80000000  Yes   No   Level 1 data cache misses

PAPI_L1_ICM  0x80000001  Yes   No   Level 1 instruction cache misses

PAPI_L2_DCM  0x80000002  Yes   Yes  Level 2 data cache misses

PAPI_L2_ICM  0x80000003  Yes   No   Level 2 instruction cache misses

PAPI_L3_DCM  0x80000004  No    No   Level 3 data cache misses

PAPI_L3_ICM  0x80000005  No    No   Level 3 instruction cache misses

PAPI_L1_TCM  0x80000006  Yes   Yes  Level 1 cache misses

PAPI_L2_TCM  0x80000007  Yes   No   Level 2 cache misses

PAPI_L3_TCM  0x80000008  Yes   No   Level 3 cache misses

PAPI_CA_SNP  0x80000009  Yes   No   Requests for a snoop

PAPI_CA_SHR  0x8000000a  Yes   No   Requests for exclusive access to shared cache line

PAPI_CA_CLN  0x8000000b  Yes   No   Requests for exclusive access to clean cache line

PAPI_CA_INV  0x8000000c  Yes   No   Requests for cache line invalidation

PAPI_CA_ITV  0x8000000d  Yes   No   Requests for cache line intervention

PAPI_L3_LDM  0x8000000e  Yes   No   Level 3 load misses

PAPI_L3_STM  0x8000000f  No    No   Level 3 store misses

PAPI_BRU_IDL 0x80000010  No    No   Cycles branch units are idle

PAPI_FXU_IDL 0x80000011  No    No   Cycles integer units are idle

PAPI_FPU_IDL 0x80000012  No    No   Cycles floating point units are idle

PAPI_LSU_IDL 0x80000013  No    No   Cycles load/store units are idle

PAPI_TLB_DM  0x80000014  Yes   Yes  Data translation lookaside buffer misses

PAPI_TLB_IM  0x80000015  Yes   No   Instruction translation lookaside buffer misses

PAPI_TLB_TL  0x80000016  No    No   Total translation lookaside buffer misses

PAPI_L1_LDM  0x80000017  Yes   No   Level 1 load misses

PAPI_L1_STM  0x80000018  Yes   No   Level 1 store misses

PAPI_L2_LDM  0x80000019  Yes   No   Level 2 load misses

PAPI_L2_STM  0x8000001a  Yes   No   Level 2 store misses

PAPI_BTAC_M  0x8000001b  No    No   Branch target address cache misses

PAPI_PRF_DM  0x8000001c  Yes   No   Data prefetch cache misses

PAPI_L3_DCH  0x8000001d  No    No   Level 3 data cache hits

PAPI_TLB_SD  0x8000001e  No    No   Translation lookaside buffer shootdowns

PAPI_CSR_FAL 0x8000001f  No    No   Failed store conditional instructions

PAPI_CSR_SUC 0x80000020  No    No   Successful store conditional instructions

PAPI_CSR_TOT 0x80000021  No    No   Total store conditional instructions

PAPI_MEM_SCY 0x80000022  No    No   Cycles Stalled Waiting for memory accesses

PAPI_MEM_RCY 0x80000023  No    No   Cycles Stalled Waiting for memory Reads

PAPI_MEM_WCY 0x80000024  Yes   No   Cycles Stalled Waiting for memory writes

PAPI_STL_ICY 0x80000025  Yes   No   Cycles with no instruction issue

PAPI_FUL_ICY 0x80000026  Yes   Yes  Cycles with maximum instruction issue

PAPI_STL_CCY 0x80000027  Yes   No   Cycles with no instructions completed

PAPI_FUL_CCY 0x80000028  Yes   No   Cycles with maximum instructions completed

PAPI_HW_INT  0x80000029  No    No   Hardware interrupts

PAPI_BR_UCN  0x8000002a  Yes   Yes  Unconditional branch instructions

PAPI_BR_CN   0x8000002b  Yes   No   Conditional branch instructions

PAPI_BR_TKN  0x8000002c  Yes   Yes  Conditional branch instructions taken

PAPI_BR_NTK  0x8000002d  Yes   No   Conditional branch instructions not taken

PAPI_BR_MSP  0x8000002e  Yes   No   Conditional branch instructions mispredicted

PAPI_BR_PRC  0x8000002f  Yes   Yes  Conditional branch instructions correctly predicted

PAPI_FMA_INS 0x80000030  No    No   FMA instructions completed

PAPI_TOT_IIS 0x80000031  No    No   Instructions issued

PAPI_TOT_INS 0x80000032  Yes   No   Instructions completed

PAPI_INT_INS 0x80000033  No    No   Integer instructions

PAPI_FP_INS  0x80000034  No    No   Floating point instructions

PAPI_LD_INS  0x80000035  Yes   No   Load instructions

PAPI_SR_INS  0x80000036  Yes   No   Store instructions

PAPI_BR_INS  0x80000037  Yes   No   Branch instructions

PAPI_VEC_INS 0x80000038  No    No   Vector/SIMD instructions (could include integer)

PAPI_RES_STL 0x80000039  Yes   No   Cycles stalled on any resource

PAPI_FP_STAL 0x8000003a  No    No   Cycles the FP unit(s) are stalled

PAPI_TOT_CYC 0x8000003b  Yes   No   Total cycles

PAPI_LST_INS 0x8000003c  Yes   Yes  Load/store instructions completed

PAPI_SYC_INS 0x8000003d  No    No   Synchronization instructions completed

PAPI_L1_DCH  0x8000003e  No    No   Level 1 data cache hits

PAPI_L2_DCH  0x8000003f  No    No   Level 2 data cache hits

PAPI_L1_DCA  0x80000040  No    No   Level 1 data cache accesses

PAPI_L2_DCA  0x80000041  Yes   No   Level 2 data cache accesses

PAPI_L3_DCA  0x80000042  Yes   Yes  Level 3 data cache accesses

PAPI_L1_DCR  0x80000043  No    No   Level 1 data cache reads

PAPI_L2_DCR  0x80000044  Yes   No   Level 2 data cache reads

PAPI_L3_DCR  0x80000045  Yes   No   Level 3 data cache reads

PAPI_L1_DCW  0x80000046  No    No   Level 1 data cache writes

PAPI_L2_DCW  0x80000047  Yes   No   Level 2 data cache writes

PAPI_L3_DCW  0x80000048  Yes   No   Level 3 data cache writes

PAPI_L1_ICH  0x80000049  No    No   Level 1 instruction cache hits

PAPI_L2_ICH  0x8000004a  Yes   No   Level 2 instruction cache hits

PAPI_L3_ICH  0x8000004b  No    No   Level 3 instruction cache hits

PAPI_L1_ICA  0x8000004c  No    No   Level 1 instruction cache accesses

PAPI_L2_ICA  0x8000004d  Yes   No   Level 2 instruction cache accesses

PAPI_L3_ICA  0x8000004e  Yes   No   Level 3 instruction cache accesses

PAPI_L1_ICR  0x8000004f  No    No   Level 1 instruction cache reads

PAPI_L2_ICR  0x80000050  Yes   No   Level 2 instruction cache reads

PAPI_L3_ICR  0x80000051  Yes   No   Level 3 instruction cache reads

PAPI_L1_ICW  0x80000052  No    No   Level 1 instruction cache writes

PAPI_L2_ICW  0x80000053  No    No   Level 2 instruction cache writes

PAPI_L3_ICW  0x80000054  No    No   Level 3 instruction cache writes

PAPI_L1_TCH  0x80000055  No    No   Level 1 total cache hits

PAPI_L2_TCH  0x80000056  No    No   Level 2 total cache hits

PAPI_L3_TCH  0x80000057  No    No   Level 3 total cache hits

PAPI_L1_TCA  0x80000058  No    No   Level 1 total cache accesses

PAPI_L2_TCA  0x80000059  Yes   Yes  Level 2 total cache accesses

PAPI_L3_TCA  0x8000005a  Yes   No   Level 3 total cache accesses

PAPI_L1_TCR  0x8000005b  No    No   Level 1 total cache reads

PAPI_L2_TCR  0x8000005c  Yes   Yes  Level 2 total cache reads

PAPI_L3_TCR  0x8000005d  Yes   Yes  Level 3 total cache reads

PAPI_L1_TCW  0x8000005e  No    No   Level 1 total cache writes

PAPI_L2_TCW  0x8000005f  Yes   No   Level 2 total cache writes

PAPI_L3_TCW  0x80000060  Yes   No   Level 3 total cache writes

PAPI_FML_INS 0x80000061  No    No   Floating point multiply instructions

PAPI_FAD_INS 0x80000062  No    No   Floating point add instructions

PAPI_FDV_INS 0x80000063  No    No   Floating point divide instructions

PAPI_FSQ_INS 0x80000064  No    No   Floating point square root instructions

PAPI_FNV_INS 0x80000065  No    No   Floating point inverse instructions

PAPI_FP_OPS  0x80000066  No    No   Floating point operations

PAPI_SP_OPS  0x80000067  Yes   Yes  Floating point operations; optimized to count scaled single precision vector operations

PAPI_DP_OPS  0x80000068  Yes   Yes  Floating point operations; optimized to count scaled double precision vector operations

PAPI_VEC_SP  0x80000069  Yes   Yes  Single precision vector/SIMD instructions

PAPI_VEC_DP  0x8000006a  Yes   Yes  Double precision vector/SIMD instructions

PAPI_REF_CYC 0x8000006b  Yes   No   Reference clock cycles

--------------------------------------------------------------------------------

Of 108 possible events, 60 are available, of which 16 are derived.


jkwack@thetamom3:~> which papi_avail

/opt/cray/pe/papi/5.6.0.5/bin/papi_avail