At what point does ChatGPT forget the first text in a conversation?

Which is very correct

At 1000 words into the conversation, it still remembers what was said.

As we approach 2,500 words, it fails to remember, and things start falling apart.

(Please note: The test presented here was directly taken from the video available at https://www.youtube.com/watch?v=MmSMAYooRas but it can be simulated and executed by anyone on ChatGPT at chat.openai.com.)

But why exactly?

Introduction to Context Window

A context window operates by creating a sliding window over the input text, focusing on a specific word at a time. Each word is embedded in a multi-dimensional vector space, allowing the AI model to analyze its contextual associations within the given window. The size of the context window is a crucial parameter, as it determines the scope of contextual information assimilated by the...

I recently(literally 1 hour ago) came across a research paper on a new type of MLLM,
full information here
https://arxiv.org/html/2401.06071v2 and i tasked my self to write a TL;DR article on everything i read and understood from paper summarized in this in < 1 hour .
Here we go

Problem Statement:

Current MLLMs(Multi modal language models) primarily focus on global information and do not consider fine-grained local information in multi-modal data. This limitation restricts their application in tasks requiring a more detailed understanding.

For example in the image above, it only references the dog in the statement “In the foreground, there’s a happy-looking dog with a white coat sitting among various animals.” It cannot provide exact spatial coordinates of the dog in the image.

E.g.

The dog is located in the top right corner of the image, with coordinates (0.7, 0.8) to (0.9...

As of writing this , the minimum wage in Nigeria is ₦ 40,000($66.66) and 27.4 million Nigerians earn less than ₦100,000($166.66) yearly. The only reason i am writing this is for a kid out there to be inspired and make magic ✨

[0] Introduction
Hi, i’m Pleasant, a teenager in Lagos Mainland, Nigeria and i like to build products & talk about it , i developed interest for software development 3 years ago (courtesy - my ICT teacher Mr. Oloni Leke) and i only started making money this year. This is me trying to sum it up in one article.
Heads up, i am not very intelligent, so i doubt there is an IQ requirement to do this. 😄

[1] The Pandemic
I’ve never liked school, teaching so many students the same way, the same thing and grading them with the same system never made sense to me, everyone is different, there are auditory leaners, Kinaesthetic leaners e.t.c and as Albert Einstein once...