Project update + 4 news stories

Today, I have a big algorithm update for you. But first, the news:

Today ChatGPT read 1099 top news stories. After removing previously covered events, there are 4 articles with a significance score over 5.8.

[5.9] Judge hints at plans to rein in Google's Play Store — WIRED

In December, a jury found Google violated US antitrust laws through unfair Google Play Store practices. On May 23, Judge James Donato began discussing potential penalties. Epic Games seeks a ban on anti-competitive contracts and better support for rival stores. Google disputes these demands, citing security concerns and appealing the verdict.

[5.9] HPV vaccine reduces cancer risk for men — The Straits Times

A study of 3.4 million people shows HPV vaccination significantly reduces head, neck, and other HPV-related cancers in males, with vaccinated males having lower cancer rates (3.4 and 2.8 per 100,000) compared to unvaccinated males (7.5 and 6.3). Between 2011-2020, US HPV vaccination rates increased from 23.3% to 43.0%.

[5.9] (UK) Rishi Sunak to call surprise general election for July — The Guardian

Prime Minister Rishi Sunak will call an early election for 4 July 2024, following improved economic indicators. Labour, under Keir Starmer, leads by 20 points in polls and aims to end 14 years of Conservative governance.

[5.9] (US) CHIPS Act last $6 billion in semiconductor award money — CNBC

The CHIPS Act has allocated $33 billion, with $28 billion going to Intel, Taiwan Semiconductor, Samsung, and Micron, and $1.5 billion to GlobalFoundries. Four smaller companies received $392 million. With $6 billion remaining, the focus is on smaller companies across the supply chain to boost economic security. The remaining funds will be allocated by year-end.

Highly covered news with significance over 5.3:

[5.5] Ceasefire negotiations in Ukraine as Putin considers freezing conflict (CTV News + 4)

[5.4] SEC approves rule changes for ethereum ETFs, boosting crypto (Financial Times + 9)

Today I’m making one of the most significant scoring updates since the beginning of the project. But first, a little background.

Background

In the very first version of the scoring formula, there were only three parameters:

• scale — how many people the event affected,

• magnitude — how big was the effect,

• potential — how likely it is that the event will cause bigger events.

The inclusion of potential was based on this hypothetical news:

• a single person died (scale = 1, magnitude = 10)

  • from old age (potential = 1)

  • from a new, unknown disease (potential = 5+)

In the second case, there’s a possibility of a disease spreading and causing more deaths, so the potential and the overall score are higher.

But this parameter had two problems.

Problem 1: Sensitivity to speculative statements.

My assumption was that even if the possible disease spread is not mentioned, the AI would catch that. But it turned out to be wrong.

ChatGPT doesn’t “read between the lines” and just rates what is mentioned in the article. If the article only delivers facts, it evaluates those facts. And if it makes speculative statements, the potential and the final score rises. The bigger the statements — the higher the score.

Basically, that the algorithm was rewarding sensationalism.

I had the credibility parameter to punish tabloid sources but it did nothing on an article level: articles from a single source had the same credibility value.

Problem 2: High correlation with scale and magnitude.

It took me a long time to notice that potential was always close to scale and magnitude. I almost never saw them diverge. So I checked the correlation between them and it turned out to be incredibly high: 0.85 with scale, and 0.92 with magnitude.

The potential didn’t really bring much insight into the news’s significance, and mostly just amplified the parameters we already had.

Solution

After these problems became clear the solution was simple — remove the potential parameter.

But I wanted to go a little further.

Some readers noticed that since the last algorithm update (the introduction of positivity parameter), the feed became skewed towards science advancements. Sure, they are often significant, but the predictions “this discovery might revolutionize the field” were used too often and too freely.

To fix this, I’m adding a new parameter that I see as the opposite of potential:

Probability — how likely it is that events speculated in the news will actually occur.

It took me a while to formulate it well to AI, but the final results seem promising. Here’s some examples:

• Futurist Marshall Brain discusses the threat of climate change and the potential collapse of civilization (probability 3/10)

• Vladimir Putin hinted at the possibility of resuming nuclear testing (probability 6/10)

• The US Food and Drug Administration has approved two vaccines to protect against the RSV (probability 10/10)

The absolute values likely aren’t precise, but they don’t really matter.

It’s the relative values that will improve the selection: hide the news that only paint an excessively grim (or bright) picture of the future, and bring up the ones that describe facts.

If you have some thoughts about the update, or simply want to like or share it, I also posted it on Twitter:

Otherwise, until next time!

Vadim