Longer context window doesn't mean increased reasoning capabilities (even ignoring loss in the middle phenomenon). Context window and reasoning window are separate aspects. RAG with less context is possibly still required to better align the responses.
OTOH, based on needle-in-haystack tests, both Gemini 1.5 and LWM are claiming near-perfect recall across the whole window. Would have to verify in real-world use.
Longer context window doesn't mean increased reasoning capabilities (even ignoring loss in the middle phenomenon). Context window and reasoning window are separate aspects. RAG with less context is possibly still required to better align the responses.
Good point, reasoning is orthogonal.
OTOH, based on needle-in-haystack tests, both Gemini 1.5 and LWM are claiming near-perfect recall across the whole window. Would have to verify in real-world use.