Coder Legion - Recent posts tagged cuda

Reverse-Engineering NVIDIA: Changing a register operand

Fri, 25 Sep 2026 14:41:55 +0000

Motivation

In the previous article I have
started reverse-engineering a simple CUDA binary by modifying the value of
a pointer variable.

After identifying where the value was stored and successfully running the modified CUDA binary, it was time to find out what the corresponding assembly instructions actually looked like at the machine-code level.

Finding the immediate size

Now, to be able to determine the size and format of the instruction (or at
least the largest constant the instruction from the previous article can hold),
I started to increase the value word-by-word. Instead of simple 0xcc from the
last time, I did:

#include <stdio.h>

__global__ void test(int *a) {
    a[0] = 0xecc;
}

int main(int argc, const char* argv[]) {
    cudaError_t err;
    int *a;
    cudaMallocManaged(&a, sizeof(int));
    *a = 0x01;
    test<<<1, 1>>>(a);
    if ((err = cudaGetLastError()) != cudaSuccess)
        fprintf(stderr, "launch: %s\n", cudaGetErrorString(err));

    cudaDeviceSynchronize();
    printf("*a = %x\n", *a);

    return 0;
}

And yes, it was there:

$ xxd -e out/test-old.sm_50.cubin | grep ecc
00000690: ecc7f000 01000000 05070002 4c980780   ...............L

Now to try a bit larger value like '0xdecc':

$ xxd -e out/test-old.sm_50.cubin | grep decc

Searching for the complete 0xdecc pattern didn't find anything. Searching for
the original 0xecc did, however, and the surrounding bytes had changed:

$ xxd -e out/test-old.sm_50.cubin | grep ecc
00000690: ecc7f000 0100000d 05070002 4c980780   ...............L

And there it was. If we compare this line with the hex dump from above, we can
see that we have an extra d in 0100000d. So I tried increasing the word
size in xxd:

$ xxd -e -g 8 out/test-old.sm_50.cubin | grep decc
00000690: 0100000decc7f000 4c98078005070002   ...............L

Now the bytes looked much more like a structured 64-bit value rather than an
arbitrary sequence of bytes. The next thing to do was simply increase the test
value and see how much the apparent immediate field could hold.
The result with the largest value was:

$ xxd -e -g 8 out/test-old.sm_50.cubin | grep ffffffff
00000690: 010ffffffff7f000 4c98078005070002   ...............L

After that, it was impossible to fit more into an int value as nvcc
complained about truncation. I naturally tried to find out what will
happen if I use uint64_t instead of int:

$ xxd -e -g 8 out/test-old.sm_50.cubin | grep ffffffff -C1
00000680: 001fc400fe4007f1 010eeeeeeee7f004   ..@.............
00000690: 010ffffffff7f005 eedd200000070204   ............. ..
000006a0: 001f9c00fde007ef 50b0000000070f00   ...............P

So we discovered something. We now had two apparent MOVs and I wasn't sure
whether they differed because they targeted different registers or because they
loaded the high and low parts of a value.

Adding a static variable

After seeing how the MOV behaved while changing the value, I tried a little
experiment. I added one more variable with a lot of bytes:

#include <stdint.h>

__device__ uint64_t x = 0xdaec;

__global__ void test(uint64_t *a) {
        x = 0xccccccccdddddddd;
        a[0] = 0xaaaaaaaabbbbbbbb;
}

After looking for my usual patterns, I got:

$ xxd -e -g 8 out/test-old.sm_50.cubin | grep bbbb -B5 -A7
00000780: 001fc400fe2007f6 4c98078000870001   .. ............L
00000790: 010000000007f002 010000000007f003   ................
000007a0: 001fc000fe4007f1 010dddddddd7f008   ..@.............
000007b0: 010cccccccc7f009 4c98078005070004   ...............L
000007c0: 001fc400fc2007f1 eedd200000070208   .. .......... ..
000007d0: 4c98078005170005 010bbbbbbbb7f006   .......L........
000007e0: 001fbc00fe2007f2 010aaaaaaaa7f007   .. .............
000007f0: eedd200000070406 50b0000000070f00   ..... .........P
00000800: 001ffc00fc4007ef 50b0000000070f00   ..@............P
00000810: 50b0000000070f00 e30000000007000f   .......P........
00000820: 001f8000fc0007ff e2400fffff07000f   ..............@.
00000830: 50b0000000070f00 50b0000000070f00   .......P.......P
00000840: 000000000000daec 0000000000000000   ................

Looking at this we can filter some of these out. We can filter by patterns I
created. Also, I noticed that the line beginning with 0xeed appeared a lot
in my dumps. Here it appeared two times, so I filtered useful stuff out:

$ xxd -c 8 -e -g 8 out/test-old.sm_50.cubin | grep 'bbbb\|cccc\|aaaa\|dddd\|eedd'
000007a8: 010dddddddd7f008   ........
000007b0: 010cccccccc7f009   ........
000007c8: eedd200000070208   ..... ..
000007d8: 010bbbbbbbb7f006   ........
000007e8: 010aaaaaaaa7f007   ........
000007f0: eedd200000070406   ..... ..

The interesting part was that the final byte changed systematically as I introduced different values. This made it look increasingly likely that the final byte represented a register operand.

This is current test output:

$ ./test-old 
*a = aaaaaaaabbbbbbbb

Can we make it display the value of variable x? Let's try hacking 0xeed2
instructions. My hypothesis was that the low bits of these instructions encode
the register operand, similarly to what we observed in the 0x10 lines:

$ xxd -c 8 -e -g 8 test-old | grep eedd2
0008a848: eedd200000070208   ..... ..
0008a870: eedd200000070406   ..... ..

I used bvi to change the final byte of the second instruction from 06 to
08:

xxd -c 8 -e -g 8 test-old | grep eedd2
0008a848: eedd200000070208   ..... ..
0008a870: eedd200000070408   ..... ..

And (not) surprisingly, I got:

$ ./test-old
*a = ccccccccdddddddd

We actually switched registers from which the value for a[0] was loaded.

Conclusion

For now we can identify (and modify to a certain extent) two instructions:

The first instruction appears to be a MOV-like instruction, with a format that
currently looks like:

010VVVVVVVV7f00R

The VVVVVVVV appears to represent an immediate value, while R appears to
identify the destination register.

The second instruction appears to select a register containing the value that is
ultimately written to a[0]. Its encoding currently appears to be:

eedX200000070Y0R

where R appears to identify the register. X/Y remain unknown; I suspect one
of them encodes information about the operand size, since I observed different
values when experimenting with 32-bit int versus 64-bit uint64_t values.

GPU-Accelerating MSCRED with CUDA, im2col, GEMM, and a Custom PyTorch Extension

Fri, 11 Sep 2026 18:07:01 +0000

Introduction

During my M.Tech in Data Science and Artificial Intelligence at IIT Bhilai (2021–2023), I conducted thesis research on multivariate time-series anomaly detection and GPU acceleration using industrial process signals.

My thesis, titled “Anomaly Detection of Multivariate Time Series Data and Acceleration Using GPUs,” explored an MSCRED—Multi-Scale Convolutional Recurrent Encoder-Decoder—and ConvLSTM-based pipeline for modelling relationships between signals and detecting abnormal temporal patterns.

Although the model produced useful anomaly-detection results, its convolution-heavy encoder created a significant CPU bottleneck. I therefore implemented GPU-accelerated operations using CUDA, transformed convolution into an im2col plus GEMM workflow, investigated global-memory and shared-memory implementations, and integrated the accelerated operations with PyTorch.

In the measured workload, encoder runtime decreased from 300 seconds on the CPU to 23.664 seconds on the GPU, representing an approximately 12.68× encoder speed-up.

Research attribution: MSCRED and ConvLSTM are established architectures introduced by the researchers cited in the References section. This article describes how I used, adapted, profiled, and accelerated selected operations as part of my M.Tech thesis.

Data confidentiality: The original research involved industrial process data. This article discusses the architecture, optimisation methodology, and aggregate benchmark results without publishing proprietary signals, internal process details, confidential datasets, or restricted source code.

The Anomaly-Detection Problem

Industrial systems often generate many related sensor and process-control signals at the same time. A single signal may appear normal when viewed independently, while the relationship between several signals may indicate an abnormal operating condition.

This makes multivariate time-series anomaly detection more difficult than analysing one signal at a time.

The model must learn:

Relationships between multiple signals
Changes across different time windows
Temporal dependencies between successive observations
Deviations from normal operating patterns
Reconstruction differences that may indicate anomalies

The goal of the research was to detect unusual multivariate patterns while preserving the temporal and spatial relationships present in the data.

MSCRED and ConvLSTM Architecture

MSCRED models relationships between multiple time-series signals by constructing signature matrices over different temporal windows.

At a high level, the pipeline contains:

Signature-matrix generation from multivariate signals
A convolutional encoder for spatial feature extraction
ConvLSTM layers for temporal modelling
A decoder that reconstructs expected signal relationships
Reconstruction-error analysis for anomaly scoring

The encoder processes the signature matrices through convolutional operations, while ConvLSTM captures how the encoded representations evolve over time.

Multivariate signals
        ↓
Signature matrices
        ↓
CNN encoder
        ↓
ConvLSTM
        ↓
Decoder
        ↓
Reconstruction error
        ↓
Anomaly score

Why Signature Matrices Are Useful

A signature matrix represents pairwise relationships between signals over a selected time interval.

By generating these matrices at multiple scales, the model can observe both short-term and longer-term relationships. This helps identify anomalies that may not be visible in an individual signal.

Role of the Encoder

The convolutional encoder extracts spatial patterns from the signature matrices. It reduces the raw matrix representation into compact feature maps that capture important relationships between signals.

Role of ConvLSTM

ConvLSTM models how those spatial representations change over time.

Unlike a standard fully connected LSTM, ConvLSTM preserves the spatial structure of feature maps while learning temporal dependencies.

Role of the Decoder

The decoder reconstructs the expected signature matrices.

When the reconstructed output differs significantly from the observed input, the reconstruction error can indicate an anomalous pattern.

Anomaly-Detection Evaluation

The purpose of the MSCRED and ConvLSTM pipeline was to detect abnormal relationships and temporal behaviour across multiple industrial process signals.

The model learned normal multivariate patterns by reconstructing signature matrices. During evaluation, the difference between the observed signature matrices and the reconstructed outputs was used to calculate a reconstruction error.

Observed signature matrix
            ↓
MSCRED reconstruction
            ↓
Reconstruction error
            ↓
Thresholding
            ↓
Normal or anomalous

A larger reconstruction error indicated that the observed relationships differed from patterns learned during normal operation and could therefore represent an anomaly.

The thesis evaluated anomaly-detection quality using the F1 score. Under the reported experimental setup, the best recorded F1 score was 0.941.

This result represents model-quality evaluation and should be interpreted separately from the CUDA runtime measurements presented later. The GPU memory optimisations were designed to reduce computation time, not to improve predictive accuracy.

Where the Performance Bottleneck Appeared

The encoder repeatedly applied convolutional operations to signature matrices.

On the CPU, these operations involved substantial nested computation across:

Input channels
Output filters
Spatial positions
Kernel dimensions
Batch elements

This made the encoder a strong candidate for GPU acceleration because many convolution calculations could be performed in parallel.

The objective was not to redesign the anomaly-detection model. Instead, it was to preserve its expected computational behaviour while accelerating selected expensive operations.

Transforming Conv2D into `im2col` and GEMM

A convolution repeatedly applies a kernel to overlapping regions of an input feature map.

The im2col, or image-to-column, transformation rearranges those local regions into columns of a matrix.

After this transformation, convolution can be expressed as a general matrix multiplication:

Output matrix = Weight matrix × im2col(Input)

This conversion is useful because matrix multiplication is highly parallel and can be executed efficiently on a GPU.

Conceptual Transformation

Input feature map
        ↓
Extract overlapping kernel windows
        ↓
Arrange windows as matrix columns
        ↓
Multiply by reshaped filter matrix
        ↓
Reshape result into output feature map

Why Use GEMM?

GEMM stands for General Matrix-Matrix Multiplication.

Modern GPU architectures are designed to perform large numbers of multiply-and-accumulate operations in parallel. Expressing convolution as GEMM allows the workload to use that parallelism.

Trade-Off

im2col introduces additional memory usage because overlapping input regions are copied into an intermediate matrix.

The method is most effective when the computational benefit of parallel matrix multiplication outweighs the cost of creating and storing the intermediate representation.

CUDA Kernel Design

The CUDA implementation divided the computation across many GPU threads.

Each thread was responsible for part of the output, while thread blocks grouped related work.

Important kernel-design considerations included:

Mapping output elements to threads
Selecting thread-block dimensions
Coalescing memory access where possible
Reducing unnecessary global-memory operations
Avoiding excessive synchronisation
Preserving correct tensor indexing
Managing intermediate buffers efficiently

The implementation required careful validation because incorrect indexing in GPU code can produce plausible-looking but incorrect outputs.

Global Memory and Shared Memory

The first GPU implementation relied primarily on global memory.

Global memory provides access to the complete input, but its latency is considerably higher than that of on-chip shared memory.

The optimised implementation staged reusable tiles inside shared memory. Threads within the same block could then reuse nearby values without repeatedly loading them from global memory.

The shared-memory optimisation was intended to:

Reduce repeated global-memory reads
Improve data locality
Increase data reuse within each thread block
Reduce memory-related kernel overhead
Improve matrix-multiplication efficiency

Conceptual Data Flow

Global memory
      ↓
Load reusable tile
      ↓
Shared memory
      ↓
Threads perform repeated calculations
      ↓
Write result to global memory

Shared memory is limited in capacity, so tile dimensions and thread-block organisation must be selected carefully.

Poorly selected configurations can:

Reduce occupancy
Increase synchronisation overhead
Create bank conflicts
Use too much shared memory per block
Limit the number of active thread blocks

Integrating the CUDA Implementation with PyTorch

To use the accelerated operations inside the existing model pipeline, I connected the CUDA implementation to PyTorch through a custom extension.

The integration enabled PyTorch tensors to be passed to the CUDA implementation and the computed results to be returned to the model pipeline.

Important integration concerns included:

Tensor dimensions
Device placement
Data types
Contiguous-memory assumptions
Compilation and linking
Error handling
CPU-versus-GPU output comparison

Correctness validation was essential. A faster kernel is not useful if it changes tensor shapes, produces incorrect indexing, introduces unacceptable numerical differences, or behaves inconsistently for the tested configurations.

The custom extension made it possible to retain PyTorch for model development while using lower-level CUDA operations for selected performance-critical components.

Experimental Setup

The benchmark used the following environment:

GPU: NVIDIA RTX A6000, 48 GB
CPU: 16-core Intel Xeon Gold 5218
System memory: 64 GB
Data points: 36,000
Number of features: 30
Batch size: 128
CPU threads: 32

Timing Methodology

The timings reported here are measurements from my thesis experiments using the specified hardware and workload. The CPU and GPU implementations used the same input dimensions and batch configuration.

The results should be treated as comparative experimental measurements rather than universal library benchmarks. Performance may vary with implementation details, CUDA configuration, tensor dimensions, data-transfer costs, compiler settings, and hardware.

Performance Benchmark Results

Component	CPU	GPU: global memory	GPU: shared memory
Encoder	300 s	27.492 s	23.664 s
ConvLSTM	1,200 s	909.208 s	847.260 s
Total runtime	2,700 s	1,594 s	1,481 s

Results at a Glance

Anomaly-detection quality: best recorded F1 score of 0.941
Encoder acceleration: approximately 12.68×
End-to-end acceleration: approximately 1.82×
Total runtime reduction: approximately 45.1%

The shared-memory implementation reduced encoder runtime from 300 seconds to 23.664 seconds, producing an approximately 12.68× encoder speed-up.

For the complete measured pipeline, runtime decreased from 2,700 seconds to 1,481 seconds. The total-runtime measurement represents the complete measured pipeline rather than only the encoder and ConvLSTM components shown separately.

Why the Encoder Improved More Than ConvLSTM

The convolutional encoder benefited strongly from GPU execution because its operations provided substantial data parallelism.

ConvLSTM achieved a smaller improvement because of:

Recurrent dependencies
Sequential temporal processing
Memory movement
Repeated state updates
More limited parallelism across time steps

Consequently, the 12.68× encoder acceleration did not translate into a 12.68× end-to-end speed-up.

After the encoder was accelerated, ConvLSTM became the dominant runtime bottleneck.

This is an important performance-engineering lesson: accelerating one component can move the bottleneck to another part of the system.

Correctness Validation

The CUDA implementation was compared with the reference implementation for the tested tensor shapes and configurations.

The optimisation work focused on reducing execution time while preserving the expected computational behaviour. Model-quality evaluation and runtime evaluation were treated as separate concerns.

Engineering Challenges

Several engineering challenges appeared during the implementation.

1. Correct Tensor Indexing

The CUDA kernel had to reproduce the same logical output as the reference convolution operation.

A small indexing error could generate incorrect feature maps without necessarily causing a runtime failure.

2. Memory Management

The im2col transformation created additional intermediate data.

This required careful allocation, reuse, and release of GPU memory.

3. Synchronisation

Threads sharing data through shared memory had to synchronise at appropriate points.

Missing or unnecessary synchronisation could cause incorrect results or reduced performance.

4. Kernel Configuration

Thread-block dimensions and tile sizes affected:

Occupancy
Shared-memory usage
Memory access
Work distribution
Overall kernel performance

5. PyTorch Integration

The extension had to account for tensor shapes, data types, device placement, memory layout, compilation, linking, and output comparison.

6. Measuring the Correct Bottleneck

Improving one component changed the relative cost of other components.

Profiling therefore had to be repeated after major optimisations.

Engineering Lessons

1. Optimising One Component Does Not Guarantee an Equivalent End-to-End Improvement

Once the encoder was accelerated, ConvLSTM represented a larger proportion of the remaining runtime.

2. `im2col` Makes Convolution GPU-Friendly but Introduces Memory Overhead

The transformation enables GEMM-based execution while creating a larger intermediate representation.

3. Shared Memory Is Valuable When Threads Repeatedly Reuse Nearby Data

Its benefit depends on access patterns, tile sizes, synchronisation, and thread-block configuration.

4. Profiling Must Guide Optimisation

Performance work should focus on measured bottlenecks rather than assumptions about which component is slow.

5. Correctness Is as Important as Speed

GPU output should be compared against a trusted reference implementation before performance improvements are accepted.

6. Custom PyTorch Extensions Offer Flexibility at the Cost of Additional Complexity

Device handling, tensor shapes, data types, compilation, numerical validation, and fallback behaviour all require careful attention.

Frequently Asked Questions

What is MSCRED?

MSCRED stands for Multi-Scale Convolutional Recurrent Encoder-Decoder.

It is an architecture designed to model relationships between multiple time-series signals and detect abnormal behaviour through reconstruction error.

What does `im2col` do?

im2col rearranges overlapping convolution windows into matrix columns, allowing convolution to be expressed as matrix multiplication.

Why did shared memory help?

Shared memory allowed threads in the same block to reuse data without repeatedly loading it from slower global memory. This reduced memory-access overhead for reusable tiles.

Did GPU memory optimisation improve the F1 score?

That was not the purpose of the optimisation, and this article does not claim that it did.

The best recorded F1 score describes anomaly-detection quality in the thesis experiments. The CPU and GPU timing results describe computational performance. They should be interpreted separately.

Why was the encoder speed-up much larger than the total speed-up?

After the encoder became faster, ConvLSTM represented a larger share of total runtime. The overall system was therefore limited by the remaining bottleneck.

Is a Custom CUDA Kernel Always Faster Than PyTorch Operations?

No.

Optimised PyTorch and CUDA libraries are already highly efficient for many operations. A custom kernel is most useful when:

The workload has a specialised access pattern
Existing operations create unnecessary overhead
Profiling identifies a clear bottleneck
Correctness and maintenance costs are justified

Can the Original Industrial Dataset Be Shared?

No.

The original dataset contains industrial process information. A public reproduction would require synthetic or appropriately anonymised multivariate time-series data.

Are These Benchmark Results Universal?

No.

Performance depends on GPU architecture, CPU configuration, tensor dimensions, batch size, kernel implementation, memory layout, CUDA and PyTorch versions, compiler settings, and workload characteristics.

The reported numbers apply to the specific experimental setup described in this article.

Limitations and Data-Privacy Considerations

The reported results were obtained on a specific hardware configuration and workload.

Different GPUs, tensor dimensions, batch sizes, memory layouts, CUDA versions, and kernel configurations may produce different results.

The original dataset cannot be published because it contains industrial process information. A public reproduction would therefore require synthetic or appropriately anonymised multivariate time-series data.

This article presents aggregate benchmark measurements and technical methodology rather than the complete industrial dataset or restricted implementation.

Future Work

Future work could focus on:

Profiling and optimising the ConvLSTM component
Reducing intermediate memory allocation
Reducing host-to-device and device-to-device memory movement
Comparing the custom implementation with optimised CUDA libraries
Evaluating additional tile and thread-block configurations
Testing across different GPU architectures
Reproducing the workflow using synthetic public data
Adding automated correctness and regression tests
Measuring kernel-level execution using GPU profiling tools

Conclusion

This work was completed as part of my M.Tech thesis research at the Indian Institute of Technology Bhilai.

It demonstrated how low-level GPU optimisation can be integrated into a deep-learning anomaly-detection pipeline.

The convolutional encoder benefited substantially from:

CUDA parallelisation
im2col-based lowering
GEMM
Shared-memory data reuse
Custom PyTorch integration

Encoder runtime decreased from 300 seconds to 23.664 seconds in the measured workload.

The broader result also illustrated an important systems principle: once one stage is accelerated, another stage may become the new performance bottleneck.

In this case, ConvLSTM limited the end-to-end improvement even after the encoder achieved a substantial speed-up.

The work reinforced that successful performance engineering requires both low-level optimisation and system-level measurement.

References

Zhang, C., Song, D., Chen, Y., Feng, X., Lumezanu, C., Cheng, W., Ni, J., Zong, B., Chen, H., and Chawla, N. V. (2019). A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 1409–1416. https://doi.org/10.1609/aaai.v33i01.33011409
Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., and Woo, W.-C. (2015). Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. Advances in Neural Information Processing Systems 28, 802–810. https://proceedings.neurips.cc/paper_files/paper/2015/hash/07563a3fe3bbe7e3ba84431ad9d055af-Abstract.html
NVIDIA. CUDA C++ Programming Guide. https://docs.nvidia.com/cuda/cuda-c-programming-guide/
PyTorch. Custom C++ and CUDA Operators. https://docs.pytorch.org/tutorials/advanced/cpp_custom_ops.html

Citation and Feedback

If this article informs academic or technical work, please cite the original MSCRED and ConvLSTM papers above.

To refer specifically to my thesis implementation, optimisation process, or reported benchmarks, you may cite this article as:

Vutnoor, R. (2026). “GPU-Accelerating MSCRED with CUDA, im2col, GEMM, and a Custom PyTorch Extension.” DEV Community.

If you found the article useful, a reaction, comment, or constructive technical question on DEV would be appreciated.

I am Ranjith Vutnoor, an AI/ML Software Engineer and IIT Bhilai alumnus working on production RAG systems, LLM evaluation, model fine-tuning, PyTorch, and GPU-accelerated machine learning.

Reverse-Engineering NVIDIA: Modifying a CUDA binary

Sun, 06 Sep 2026 18:41:28 +0000

Motivation

For quite a while I wanted to find out how NVIDIA really works under the hood.
One of the greatest unknowns for me was how GPU code is compiled, packaged and
loaded. So I accepted my own challenge: let's try and decode machine code
running on GPU.

Test code

I needed to start really simple and to get as much data as possible. My first
CUDA program to try and decipher was simply:

#include <stdio.h>

__global__ void test(int *a) {
        a[0] = 0x00;
}

int main(int argc, const char* argv[]) {
        cudaError_t err;
        int *a;
        cudaMallocManaged(&a, sizeof(int));
        *a = 0x01;
        test<<<1, 1>>>(a);
        if ((err = cudaGetLastError()) != cudaSuccess)
                fprintf(stderr, "launch: %s\n", cudaGetErrorString(err));

        cudaDeviceSynchronize();
        printf("*a = %x\n", *a);

        return 0;
}

I won't get into too many details here (I am not a CUDA expert), but essentially, the function marked with __global__ is the kernel that will run on the device. The cudaMallocManaged() function takes care of allocating memory that can be accessed by both the host and the device, saving us some additional memory-management code. The test<<<1, 1>>>(a) syntax launches the test kernel with one block containing one thread, while the CUDA runtime takes care of loading the kernel for execution on the GPU. Finally, cudaDeviceSynchronize() waits for the GPU to finish executing the kernel, ensuring that the result written to a is available when we access it from the host.

Build

I also created a reusable build command that will also give me something to
start with:

nvcc -arch=sm_50 -lineinfo --keep --keep-dir out -o test test.cu

Note: for this to work, the out directory needs to be created before running
nvcc compiler. Now I got a folder full of really useful files:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ ls out/
test.cpp1.ii    test.cudafe1.cpp     test_dlink.fatbin    test_dlink.reg.c        test.fatbin.c   test.ptx
test.cpp4.ii    test.cudafe1.gpu     test_dlink.fatbin.c  test_dlink.sm_50.cubin  test.module_id  test.sm_50.cubin
test.cudafe1.c  test.cudafe1.stub.c  test_dlink.o         test.fatbin             test.o

The test.sm_50.cubin file sounded like exactly what I was looking for:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ file out/test.sm_50.cubin 
out/test.sm_50.cubin: ELF 64-bit LSB executable, NVIDIA CUDA architecture,, statically linked, not stripped

First hack

Now I wanted to try changing the value of a in the test function and see how
cubin file will behave. I had no idea what I was dealing with, so I started
checking the file with random small values to get the one with least repetitions
in the CUDA binary. After some random attempts I realized that the best one is:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ hexdump out/test.sm_50.cubin | grep bc
0000680 07f1 fde0 bc00 001f 02ff 0007 2000 eedc
00008f0 0000 0000 0000 0000 04bc 0000 0000 0000

So I decided to change my test function to:

__global__ void test(int *a) {
        a[0] = 0xbc;
}

If everything went well, I would see an additional line here containing my bc.
The reason I started with really small values is because I had no idea what kind
of byte ordering I will encounter. After rebuilding the binary, I now got:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ hexdump out/test.sm_50.cubin | grep bc
0000310 f1bc 06d4 02fc 0000 0209 0000 0000 0000
0000690 f000 0bc7 0000 0100 0002 0507 0780 4c98
00006a0 07f2 fe20 bc00 001f 0003 0517 0780 4c98

Well, I definitely got a bit more different hex dump. So I tried just changing
it again, this time to 0xcc (just randomly) to see what I will get. And I got:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ hexdump out/test.sm_50.cubin | grep cc
0000690 f000 0cc7 0000 0100 0002 0507 0780 4c98

This was promising. Notice that this is the same as address 0x0000690 in the
previous run, except we got 0cc7 instead of 0bc7. Now I could try and find
this kind of pattern in the real binary:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ hexdump test | grep 0cc7
000cc70 528b 4830 e789 5c89 1024 c748 2444 0008
008a700 0001 0087 0780 4c98 f000 0cc7 0000 0100

We can see our value on address 0x008a700. Right now, if we run the binary,
we get:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ ./test 
*a = cc

Now, I used bvi to quickly find the address and changed 0cc7 to 0dd7.
So the result was:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ hexdump test | grep 0dd7
000dd70 01bb 0000 e900 ff76 ffff 0f66 441f 0000
008a700 0001 0087 0780 4c98 f000 0dd7 0000 0100

Well, hopefully, I didn't corrupt the binary. When I tried running it:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ ./test 
*a = dd

Bingo. We didn't recompile the CUDA source, regenerate the CUBIN, or even touch
the host executable's source code. We changed bytes inside the compiled binary,
and the GPU executed the modified instruction.

Final thoughts

We managed to modify code running on the GPU without recompiling it. More
importantly, we have found what appears to be an instruction field containing
our constant. Given what the kernel does, it is likely part of a mov-like
instruction, although we haven't decoded it yet.

So the next question is obvious: what is the real encoding of the instruction?

Coder Legion - Recent posts tagged cuda

Reverse-Engineering NVIDIA: Changing a register operand

Motivation

Finding the immediate size

Adding a static variable

Conclusion

GPU-Accelerating MSCRED with CUDA, im2col, GEMM, and a Custom PyTorch Extension

Introduction

The Anomaly-Detection Problem

MSCRED and ConvLSTM Architecture

Why Signature Matrices Are Useful

Role of the Encoder

Role of ConvLSTM

Role of the Decoder

Anomaly-Detection Evaluation

Where the Performance Bottleneck Appeared

Transforming Conv2D into im2col and GEMM

Conceptual Transformation

Why Use GEMM?

Trade-Off

CUDA Kernel Design

Global Memory and Shared Memory

Conceptual Data Flow

Integrating the CUDA Implementation with PyTorch

Experimental Setup

Timing Methodology

Performance Benchmark Results

Results at a Glance

Why the Encoder Improved More Than ConvLSTM

Correctness Validation

Engineering Challenges

1. Correct Tensor Indexing

2. Memory Management

3. Synchronisation

4. Kernel Configuration

5. PyTorch Integration

6. Measuring the Correct Bottleneck

Engineering Lessons

1. Optimising One Component Does Not Guarantee an Equivalent End-to-End Improvement

2. im2col Makes Convolution GPU-Friendly but Introduces Memory Overhead

3. Shared Memory Is Valuable When Threads Repeatedly Reuse Nearby Data

4. Profiling Must Guide Optimisation

5. Correctness Is as Important as Speed

6. Custom PyTorch Extensions Offer Flexibility at the Cost of Additional Complexity

Frequently Asked Questions

What is MSCRED?

What does im2col do?

Why did shared memory help?

Did GPU memory optimisation improve the F1 score?

Why was the encoder speed-up much larger than the total speed-up?

Is a Custom CUDA Kernel Always Faster Than PyTorch Operations?

Can the Original Industrial Dataset Be Shared?

Are These Benchmark Results Universal?

Limitations and Data-Privacy Considerations

Future Work

Conclusion

References

Citation and Feedback

Reverse-Engineering NVIDIA: Modifying a CUDA binary

Motivation

Test code

Build

First hack

Final thoughts

Transforming Conv2D into `im2col` and GEMM

2. `im2col` Makes Convolution GPU-Friendly but Introduces Memory Overhead

What does `im2col` do?