Significant scoring update + COP28's net zero pledge and Russia's heavy toll in Ukraine conflict

Hey everyone! Yesterday I made a significant update to the scoring formula. Read about it after the news.

Today ChatGPT read 1223 top news stories. After removing previously covered events, there are 3 articles with a significance score over 7.

[8.2] COP28 agreement aims for net zero by 2050 — CNBC

Government ministers from nearly 200 countries at COP28 in Dubai agreed to a deal calling for a transition away from fossil fuels, aiming to achieve net zero by 2050. The proposal includes tripling renewable energy capacity and doubling energy efficiency improvements by 2030. The agreement did not mandate an absolute phase-out of hydrocarbons. Reactions are mixed, with some hailing it as historic while others express disappointment over the lack of a phase-out mention.

[7.8] Gaza economy devastated by conflict, World Bank provides aid — The Guardian

The World Bank reported that the Gaza economy is at 16% of its capacity due to the Israel-Hamas conflict, leaving 85% of workers jobless. The Bank is providing $20m for food and medical aid. 60% of communications infrastructure, health and education facilities, and 70% of commerce infrastructure are damaged. Over half a million are homeless, and poverty has increased.

[7.8] Russia has suffered significant losses in the Ukraine war — Reuters

A declassified U.S. intelligence report reveals that Russia has suffered 315,000 dead and injured troops in the Ukraine war, representing nearly 90% of its initial personnel. This has set back Russia’s military modernization by 18 years. Russia has been forced to relax recruitment standards and deploy older civilians due to significant losses.

Now, the update.

Over the last month I noticed a lot of Reuters articles making the top. The reason was the credibility rating — ChatGPT gave Reuters the highest credibility rating available — 9.5/10.

The formula I used made it too hard for articles from other sources to get the top spot — the weight of credibility was simply too high. Even articles from sources with a credibility 9/10 were getting a final significance score that often was too low to compete with Reuters.

The fix was simple — I reduced the weight of the credibility rating. That means that from now on, the article content will have more weight in the score than the source credibility. The sources making the top will be more diverse and top articles will be more significant. There is a small cost: it’s now a little more likely that you’ll read an article from Daily Mail.

The second change is not as straightforward and even a bit controversial.

One of the most frequent requests I got since the start of the project was to have a way to surface positive news. But I was against it. I had two slightly overlapping arguments:

• Each event can be a combination of positive and negative effects. Seemingly good events can have bad consequences: AI advances increase productivity but may lead to mass unemployment. And vice versa: increasing interest rates lead to layoffs but prevent inflation.

• Categorization is very subjective, as people have different values. Local factory closes: good for the environment and public health, bad for the economy and factory workers.

I still think the arguments stand, but over time the question of positivity changed for me. It went from “What is the combined effect of the event on the humanity?” to “What feeling am I left with after reading the article?”.

I felt like the top articles were mostly negative, but I didn’t have a proof until now.

I asked ChatGPT to give me positivity rating for each article over several days and tracked the results.

«The Results Will SHOCK You!». It turned out, after certain threshold, the higher the article significance was, the more likely it was that it had a negative tone:

Disasters, destruction and deaths — the formula was giving these news the highest possible ratings.

One of the passionate readers described this problem very eloquently:

All this lead me to making probably the most controversial change in the algorithm so far: from now on, the formula will punish news for negativity and reward it for positivity.

I understood that this could jeopardize the scoring. Positivity doesn’t really have much to do with significance — we don’t want to read just the “feels good” news. The existence of news about saved puppies was one of the reasons I started the project in the first place.

That’s why I was very conservative with the weight of the positivity parameter — initial attempts to introduce it led to a feed that was skewed too much on positive side. I lowered the weight several times until I found what I believe was the optimal value: positivity has 5 times lower effect on the score than the scale and magnitude. Still, even that minor correction was enough to fix the bias:

If you’ve read so far, you’re probably as excited about these changes as I am. I want to thank you all for supporting the project so far and hope the next chapter will be even more interesting.

Sincerely, Vadim