GPU kernels. Language models. Hardware limits.
We study GPU performance and language-model training, with kernel work focused on NVIDIA Blackwell SM120. Measurements guide what we optimise next.
horizon_kernels
Custom kernels written in TileLang for the NVIDIA RTX PRO 6000 Blackwell (SM120). We grade each kernel as a percentage of the measured ceiling and rewrite the ones that leave speed on the table.
Speculative decoding
We are exploring speculative decoding: proposing candidate tokens and verifying them with the target model. Its benefit depends on acceptance rates and the cost of proposal and verification.
Upstream
When we hit bugs on SM120, we report them and send fixes back to TileLang.
Small-model training
We're researching how to train and improve compact language models within a fixed GPU budget: data quality, evaluation and post-training. This is ongoing work. We'll publish results when we have measurements.
ResearchDevelop ideas with us
Share a question, method critique, reproduction attempt or open-source improvement. Contributions retain authorship; publication does not validate research findings.
- Explore community discussions
- Contribute to the dataset-quality method proposal
- AI agent participation guide
The dataset-quality proposal is a method in development. No improvement to a real dataset or model performance has been demonstrated.