@southernwolf @nixCraft

Strange choice of test for general mathematics capability. I wouldn't expect a transformer network to be that well suited to primality testing

But this suspected trend of #LargeLanguageModels getting worse as they are "aligned" was already acknowledged by #OpenAI in their #InstructGPT paper
https://openai.com/research/instruction-following

"it introduces an “alignment tax”: aligning the models only on customer tasks can make their performance worse on some other academic NLP tasks."

https://open-assistant.io

Please register and participate (as annotator) to the amazing
@ykilcher
project that has the goal to build a crowd sourced #opendata #opensouce alternative to proprietary #chatGPT, based on the #instructgpt logic, with much more!

#OpenAssitant #RLHF

https://www.youtube.com/watch?v=64Izfm24FKA

This script extracts text from a given file/URL and splits into sections. It writes extracted text and each section to a separate text file. It writes out the #AI-generated summary for each subsection, and a combined section summary if needed.

It splits into subsections of 1000-3000 tokens based on HTML section headings or numbered section headings. It uses #InstructGPT (text-davinci-003) to generate section summaries, and then summarizes the lower-level summaries to produce an overall summary.

Given the enormous attention people have been given to #ChatGPT in the last few days, it's fun to read the lukewarm reception of the technical innovations and data collection for the #InstructGPT paper it is based on.

(Although it did still get a solid accept for #Neurips22, of course)

https://openreview.net/forum?id=TG8KACxEON