Are news headlines enough for stock sentiment analysis?

TL;DR

Our system read "Hummer is making a comeback, but this time it's electric" and returned no stock. The saved reference linked Hummer to General Motors (GM). That concrete miss illustrates the problem: a headline may name a brand while the output needs its listed parent.

The broad GDELT test matched the complete saved company lists for 7 of 20 eligible headlines with a benchmark target. Article bodies could add useful context, but we have yet to measure that improvement. The audit also exposed conflicting model scores, matching defects and incorrect reference labels.

Watch how the Hummer headline lost General Motors

A ticker is a short code for a traded security. Apple uses AAPL; General Motors uses GM. The task is to turn a headline into a list of the stocks it concerns. Several companies can appear, while [] means that the system returned an empty list.

The animation first uses a real Apple headline to explain that task, then shows the actual Hummer failure from our GDELT test. XGBoost, a model built from decision trees, strongly linked Hummer to GM. MiniLM, a small text model, gave that link a very low score. Their saved combined score fell below the inclusion cutoff, so the system omitted GM.

The headline contains a useful brand clue. This failure gives us a specific model disagreement and company-matching problem to investigate before attributing the miss entirely to missing article text.

The source changed, and the task became harder

I am exploring inexpensive historical news coverage for NexusTrade. Headlines are appealing because they are compact and cheap to process. The intended output is an array of associated assets, with sentiment assessed separately for each one. A story can produce several tickers or an empty array.

Our full pipeline matched 355 of 373 clear, complete, in-universe Massive references. That 95.17% result covered 100 stocks plus SPY. The broader GDELT probe sampled 500 previously unused headlines from 60 cached files covering 2019 through 2026 and 417 publishers, without a company or financial-topic filter.

Only 20 headlines in that probe had complete, clear, nonempty reference arrays inside our benchmark universe. The model matched seven. Those counts describe this sampling frame and bounded registry; they do not estimate the financial-news percentage of the entire GDELT archive.

Inspect the headlines and the empty outputs

These are three actual inputs and outputs from the GDELT probe. Each headline identifies a brand; the reference maps that brand to a listed parent. The current pipeline returned an empty array for all three.

GLiNER, our name extraction component, had detected Sam's Club and Hummer above its configured name threshold. The frozen resolver nevertheless lacked entries for all three brands. The learned scorers also rejected Sam's Club and Cash App. Improving extraction alone would leave those failures unresolved.

The model combination discarded a useful prediction

Hummer showed why looking at final accuracy alone was insufficient. XGBoost gave GM a relevance score of 0.968. MiniLM, the small text-reading model paired with it, gave GM 0.010. Their fixed combination produced 0.352, below the 0.65 inclusion cutoff.

The US hiring headline showed the same pattern for SPY: XGBoost scored 0.847, MiniLM scored 0.030, and the combined score was 0.294. These are model scores, rather than calibrated probabilities.

Article text might change the inputs enough to improve both scorers. The saved results already identify another repair to evaluate: how the system handles disagreement between a strong correct component and a confidently wrong one.

Where a body could provide useful context

The GDELT references linked a Gears of War announcement to Microsoft, a Hillbilly Elegy headline to Netflix, and a Deadpool headline to Disney. The pipeline returned empty arrays. An article that states the publisher, distributor or parent company could turn those implicit relationships into explicit text.

A body could also help distinguish financial implications. A product announcement, a safety problem, an earnings beat and weaker guidance call for different judgments. Company relevance and the direction of the news are separate outputs.

The same pipeline's Massive sentiment test had 79.68% direction agreement on 443 clear reference pairs. With a confidence cutoff, it emitted 212 scores across 512 reference targets, of which 207 agreed: 97.64% agreement at 41.41% coverage. Sentiment was measured given the reference companies; successful ticker retrieval is an additional requirement.

The size of the body-text improvement remains an empirical question. None of these results compares matched headline and body inputs.

The references also contained mistakes

One helicopter-crash headline was labeled Berkshire Hathaway. The saved provider response explained that target with the words "Unrelated to this headline." The cached repeat instead identified Boeing outside our registry. The erroneous Berkshire target existed in the raw response before parsing.

Another first-pass reference linked Jerry Harris to Netflix, while the cached repeat returned no target. A headline reading "DIS Hiring Event This Week" counted as a successful Disney prediction, although the title alone leaves the abbreviation ambiguous.

Those cases affect the meaning of the 7/20 result. They remain useful diagnostics, but the references need review before we turn that fraction into a clean estimate of model quality. Retraining against every current target would preserve some evaluation mistakes.

More words also leave matching defects to repair

The resolver treated the word Beacon inside Akron Beacon Journal and Beacon Capital Partners as evidence for BECN. It then boosted those matches. MiniLM was also highly confident in both wrong associations.

An explicit Berkshire Class B headline produced both BRK.B and BRK.A. The reference contract restricted an explicit share class to the applicable security. This is an identity-resolution problem visible in a fully named headline.

A larger input may help a scorer reject some false matches. The resolver still needs to preserve the complete entity and distinguish specific share classes.

The experiment that would answer the body question

I paused the model work and inspected these saved predictions before starting more training. The useful next comparison would use the same articles and reference target policy, with one arm receiving the headline and another receiving the headline plus extracted body. Both arms would keep article groups in the same train, validation and test partitions.

The comparison should report how many bodies were successfully retrieved, because a body-only result can look stronger after difficult or unavailable pages disappear. It should measure ticker arrays, missed relevant assets, false assignments and sentiment coverage on the same population.

That test would show how much additional text buys us. The current evidence identifies enough concrete failures to design it, while leaving its outcome open.

For more of the research behind the platform, browse NexusTrade Explore.

Continue exploring