IEEE ICDM 2025 Workshops · third author
LLM-HyPZ
Published LLM-assisted pipeline for hardware vulnerability discovery.
Problem
Hardware vulnerabilities are expensive because they persist after deployment. MITRE’s 2021 hardware CWE list relied on expert Delphi ranking. Software has a data-driven CWE Top 25; hardware did not have an equivalent evidence pipeline over CVE text.
Context and constraints
CVE descriptions mix hardware, firmware, and software language, at a scale of more than 100,000 recent records. The 2021–2024 snapshot used in the paper contains 114,836 entries. Manual labeling cannot keep up, and keyword filters miss semantic overlap. LLM-HyPZ was built as a three-stage, minimally supervised mining framework and later used to narrow candidates for MITRE’s 2025 Most Important Hardware Weaknesses update.
My specific role
Graduate Research Assistant in MITHRA Lab and third author. IEEE lists the author affiliation as College of Information Science, University of Arizona. Public bios describe data preprocessing, pipeline organization, RAG-related workflows, and experimental coordination. The paper does not include a CRediT contribution table, so this page does not invent a sole-architect claim.
System architecture
CVE corpus 2021–2024
114,836 vulnerability descriptions
Stage 1 · zero-shot LLM classification
Prompted hardware vs software labels, human-in-the-loop prompt refinement
Stage 2 · embeddings + K-means
OpenAI text-embedding-3-large, 3072 dimensions; k chosen with the elbow method
Stage 3 · n-grams + LLM summarization
Frequency features plus prompt-driven cluster labels in hardware-security language
1,742 hardware CVEs · five themes
411 of 1,026 CVEs later used in MIHW 2025 analysis
Architecture taken from the IEEE/arXiv paper, not from a public application repository.
Technologies
Engineering decisions
Zero-shot before supervised training
Hardware CVE labels are scarce. The paper uses prompt-conditioned LLMs as a pre-filter instead of waiting on a large annotated set.
Embeddings over LDA
Related work used LDA and sequence labeling. Contextual embeddings plus clustering keep semantic neighborhood structure that topic models flatten.
Keep experts in the loop
Figure 3 in the paper contrasts brittle rule filters with LLM screening plus human prompt revision. That is the right control for a security corpus.
Challenges and trade-offs
Ambiguous CVE language
Entries conflate firmware, drivers, and silicon. Binary hardware/software labels are a modeling choice, not a physical truth.
Validation versus corpus scale
99.5% accuracy is reported on a curated validation set, not as a claim that every one of 114,836 records is perfectly labeled.
Closed research code
There is no public isjustabhi/LLM-HyPZ application repo. Reproducibility for outsiders depends on the paper, not a README.
Results and evidence
1,742 hardware-related CVEs
Filtered from 114,836 records (2021–2024).
Five recurring themes
Firmware/BIOS privilege escalation, memory corruption in mobile and IoT systems, physical-access exploits, and related confidentiality/availability failures, as labeled in the paper.
Downstream MIHW 2025 support
The pipeline surfaced 411 of the 1,026 CVEs used in later MIHW analysis, reducing expert search space.
Publication
IEEE ICDM 2025 Workshops, pp. 1268–1273, doi: 10.1109/ICDMW69685.2025.00150. Also arXiv:2509.00647.
What I learned
- Security NLP is a retrieval and labeling problem before it is a chatbot problem.
- Hybrid pipelines beat end-to-end generation when the corpus is large and the labels are expensive.
- Publication is the evaluation: reviewers, a DOI, and a numbered contribution to MITRE’s process.
Potential improvements
- Release a sanitized reproduction notebook for the clustering and labeling stages.
- Publish per-author task notes so contribution is not inferred from author order alone.
- Measure calibration of the hardware/software classifier on newer CVE years.