#Earthquake (#sismo) confirmed by seismic data.⚠Preliminary info: M4.6 || 29 km S of #Vicuña (#Chile) || 3 min ago (local time 07:45:18). Follow the thread for the updates👇

1/ How robust and reliable is the code generated by #LLMs, especially for real-world software development? A recent work [2] constructed a new benchmark based on [1] to evaluate if the generated code uses API correctly. Four popular #LLMs -- #GPT3.5, #GPT4, #Llama2, and #Vicuna -- are tested, and #GPT4 under zero-shot scored 62.09% misuse rate. Even with one-shot relevant examples the misuse rate of #GPT4 is 49.17%.

#GenerativeAI #papers #NLP #NLProc #SoftwareDevelopment

Some "open AI" is much more open than the industry dominating offerings. There's #EleutherAI, a donor-supported nonprofit whose model comes with documentation and code, licensed #Apache2. There are also some smaller academic offerings: #Vicuna (UCSD/CMU/Berkeley); #Koala (Berkeley) and #Alpaca (Stanford).

These are indeed more open (though Alpaca - which ran on a laptop - had to be withdrawn because it "hallucinated" so profusely).

40/

#Earthquake (#sismo) M3.5 strikes 77 km NE of #Vicuña (#Chile) 9 min ago. More info: http://m.emsc.eu/?id=1542572

Comparing #Vicuna, an #opensource conversational model, versus proprietary models.

“The development of Vicuna demonstrates promising progress in advancing #opensource conversational #AI.

By fine-tuning the #LLaMA model architecture and training methodology specifically for dialog applications, researchers were able to create a freely available conversational agent competitive with leading proprietary alternatives”

https://hackernoon.com/vicuna-how-does-it-compare-to-llama-alpaca-and-chatgpt

#Earthquake (#sismo) M2.6 strikes 13 km NE of #Vicuña (#Chile) 22 min ago. More info: http://m.emsc.eu/?id=1529814

#Vicuna 33B has surpassed #chatGPT 3.5 in #alpacaEval leaderboard.
#ai #ml

Ever wondered how to run the Vicuna-13B LLM with GPU acceleration on an Apple Silicon machine? Check out this guide on my website! It walks you through compiling the llama.cpp port specifically for Apple Silicon, finding a GGMLv3 compatible version of Vicuna, and running the model with to utilize the GPU. #AppleSilicon #Vicuna #LLAMA #MachineLearning #AI

https://flori.dev/reads/llama-cpp-vicuna-13b-apple-silicon

Faffing around with #llama.cpp over lunchtime. At first blush, I'm getting results in the same league as #ChatGPT 3.5-turbo with the 4-bit quantized #LLaMA 7B at ~1/2 the speed. Setting up is quite seamless so far.

Can't wait to try out the 13B #vicuna model, although I'm not sure I'll have the patience to use it!

@lupyuen would be interesting to do it locally with a #llm you have control on without being dependent of #OpenAI. I’m thinking about migrating to #Vicuna or #Falcon.

https://lmsys.org/
https://falconllm.tii.ae

#Earthquake (#sismo) M3.1 strikes 28 km NE of #Vicuña (#Chile) 10 min ago. More info: https://www.emsc-csem.org/Earthquake/earthquake.php?id=1272997

Was ähnlich schlecht, wie bei #ChatGPT 3.5 läuft: das #vicuna-Modell (13B 1.1) liefert völlig veraltete Beispiele. Besonders bei WinForms werden über 11 Jahre alte Objekte genutzt, um ein UI aufzubauen. Das funktioniert unter aktuellen Windowsversionen schon nicht mehr.

Die Modelle haben keinen Kontext zu den Versionen von Programmbibliotheken. Da hilft es nicht mal, dass man weiß, dass es bis zum Datum X trainiert wurde. Alles vor Datum X wird einfach vermischt.

🔔#Earthquake (#sismo) M4.3 occurred 67 km N of #Vicuña (#Chile) 4 min ago (local time 02:35:22). More info at:
📱https://www.emsc-csem.org/service/application/
🌐https://m.emsc.eu/?id=1267123
🖥https://www.emsc-csem.org/Earthquake/earthquake.php?id=1267123

⚠Preliminary info: M4.3 #earthquake (#sismo) about 70 km N of #Vicuña (#Chile) 3 min ago (local time 02:35:22). Updates at:
📱https://www.emsc-csem.org/service/application/
🌐https://m.emsc.eu
🖥https://www.emsc-csem.org

@inthehands but the power of generative AI could also leverage people with smaller means than big corporate. I think it was too rapid the way #OpenAI rolls that out in public tho, and changed it as a for profit model. They should change their name. I prefer Open Source #LLM like #Vicuna that which is as good as #GPT3 and #GPT4.

https://lmsys.org/blog/2023-03-30-vicuna/

I don't think they will achieve #AGI tho, someday but it's too early #OpenAI are just an hype right now. It won't last.

I somehow messed up my previous environment and couldn't run my little AIs for a while.

Now, thanks to the power of conda version control* I got my little Vicuña up and running again.

*: delete everything and start over.

#AI #Vicuna

I don't think #Google is in trouble. #Vicuna is dumb. Wrong answers are worse than useless.

#llama #ai

Made a #bash function for #Vicuna for example:

vic What is the command to know my hostname

The command to know your hostname in Linux is `hostname`. You can run this command in your terminal and it will return the name of your computer or server as set in the `/etc/hostname` file. [end of text]

Not sure this is useful, but we'll see. #AI

Sir:What's the root of 144?
Vicuna: The root of 144 is 1440.
Sir:No, the square root.
Vicuna: My apologies. The square root of 144 is approximately 12.3.

#Vicuna

The #LLM lunch is leaking and getting eaten up by academia and the community, with amazing advancements within months. What fun!

https://www.semianalysis.com/p/google-we-have-no-moat-and-neither

Yes, they are still stochastic parrots, but it comforting to hear that it no longer the dominion of big tech players. Great summary on what is going on in the #NLP LLM space. Literally a zoo now:

#Llama #Alpaca #Vicuna #Koala #LORA #GPT4All