When we started VietCheckMed, the problem statement felt deceptively simple: given a Vietnamese medical advertisement, does it comply with Ministry of Health regulations? Eighteen months later, having built, broken, and rebuilt the system multiple times, I can tell you the simplicity was entirely deceptive.

This post is about what we learned — not the academic version that will appear in the AAAI paper, but the practitioner version. The design choices that worked, the ones that didn't, and the surprises that production invariably delivers.

Why this problem is harder than it looks

Medical advertisement compliance in Vietnam involves evaluating content against Circular 09/2015/TT-BYT and its amendments — a regulatory document that is itself ambiguous, inconsistently applied, and occasionally contradictory. When we asked three compliance officers to independently label the same 100 ads, their agreement rate was around 72%. That baseline matters: a system that can't beat human agreement floors is not a useful system.

The second complication is modality. Vietnamese medical ads are almost always multimodal — a combination of imagery (product photos, celebrity endorsements, clinical-looking graphics) and text (claims, disclaimers, ingredient lists). Violations frequently live in the interaction between modalities: a visually implied claim that the text never makes explicit, or a disclaimer buried in an image that's technically present but practically illegible.

The most common violation type we found was "implied superiority" — a visual claim that the product cures or significantly improves a condition, paired with text that makes no such claim. Purely text-based systems miss this entirely.

Architecture decisions

Our final architecture has three stages: retrieval, reasoning, and adjudication.

Stage 1: Retrieval

Before any LLM sees the advertisement, we retrieve the relevant regulatory clauses. We built a structured regulatory knowledge base from the circular and its amendments, chunked at the article and sub-article level, with metadata about violation type, product category, and claim type.

The retrieval strategy that worked best was a hybrid: sparse BM25 retrieval over the full regulatory text, combined with dense retrieval using a multilingual embedding model fine-tuned on Vietnamese legal text. We tried many combinations; this one gave the highest recall at reasonable latency.

One thing that surprised us: query formulation mattered enormously. Naive retrieval using the raw advertisement text as the query returned mediocre results. Using the LLM to first generate a "compliance query" — a structured description of the advertisement's claims — improved recall by ~18 points.

Stage 2: Reasoning

For compliance reasoning, we evaluated both Chain-of-Thought (CoT) and Tree-of-Thought (ToT) prompting strategies. The paper covers this in detail, but the short version: CoT was faster and generally better for clear-cut violations. ToT added meaningful signal for ambiguous cases — those "implied superiority" scenarios where the answer depended on how you weighted competing interpretations.

We ended up using a hybrid: CoT as the default path, with ToT triggered when confidence was below a threshold. This kept median latency low while improving accuracy on the hard cases.

Stage 3: Adjudication

The adjudication stage takes the reasoning outputs and produces a final compliance verdict with explanation. This is where we put the most engineering effort, and where the most production surprises lived.

The key challenge: the LLM's reasoning could be correct but its verdict wrong, or its verdict correct but its explanation legally unusable. Compliance officers needed explanations they could actually cite in enforcement actions — this required a post-processing step that mapped model outputs back to specific regulatory clauses.

What production taught us

We built an excellent system in the lab. Then we deployed it against real ad intake workflows at two regulatory bodies, and discovered several things our evaluation set hadn't captured:

  • Image quality variance. Lab images were clean product shots. Production images included scanned documents, social media screenshots, compressed photos, and images with overlaid text in non-standard fonts. OCR quality degraded significantly on these. We needed a separate preprocessing pipeline.
  • Regulatory drift. The circular gets amended. Our knowledge base needed versioning. This sounds obvious in retrospect.
  • Confidence calibration. Our model was overconfident on Vietnamese health product categories it had seen little training data for. We added calibration layers and mandatory human review for low-data product categories.
  • Latency vs. accuracy tradeoffs. Under real intake load, the adjudication stage was the bottleneck. We had to compromise on ToT usage to hit acceptable throughput targets.

The dataset problem

We built VietAestheticAds-MM — the dataset introduced in the paper — because nothing adequate existed. Creating it took longer than building the model. Getting regulatory experts to label 4,000+ advertisements consistently required multiple calibration sessions, a detailed annotation guide, and a disagreement resolution protocol.

I'm sharing this because I think dataset creation is dramatically underweighted in how we think about AI research timelines. The model took three months to build and refine. The dataset took five.

What's next

The current system handles image + text advertisements. The next frontier is video — pharmaceutical TV and social media video ads that combine moving imagery, voiceover, text overlays, and celebrity appearances into a single compliance decision. The regulatory framework for these is less settled, which is both a research opportunity and a genuine societal problem.

We're also exploring generalization. VietCheckMed was built for Vietnamese MoH regulations, but the architecture should generalize. MM-RegCheck (currently in rebuttal at CIKM 2026) tests this hypothesis across three regulatory domains and two languages. Early results are encouraging.

The full paper will be available at AAAI 2026. The VietAestheticAds-MM dataset will be released upon publication. Questions welcome at [email protected].