I had a conversation with ⁠Ayomide Odumakinde⁠ (member of technical staff at Cohere AI labs) . We discussed artificial intelligence, the history of ideas in this field, current tools and methods, and what to expect in the years to come.

Links to things discussed/mentioned in this episode

Text Tokenization

https://www.ai21.com/knowledge/tokenization/

https://huggingface.co/learn/llm-course/en/chapter2/4

Supervised Finetuning:

https://huggingface.co/learn/llm-course/en/chapter11/1

Scaling law papers

⁠Kaplan et al, 2020⁠

⁠Hoffmann et al., 2022⁠

⁠Sardana et al., 2024⁠

Reinforcement learning

https://en.wikipedia.org/wiki/Reinforcement_learning

Reinforcement learning through human feedback (RLHF)

⁠Ziegler, et al, 2019⁠

⁠Learning to summarize with human feedback⁠

Reinforcement Learning with Verifiable Rewards

https://allenai.org/blog/tulu-3-technical

Effective context engineering & context rot

https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents

Qroq inference engine

https://groq.com/

Continual AI & learning

https://www.continualai.org/

https://www.ibm.com/think/topics/continual-learning

Methods for Agent augmentation

⁠Retrieval-augmented generation (RAG)⁠

⁠Function & tool calling with LLMs⁠

⁠Long-term memory in agentic systems⁠

Follow us on:

Twitter ⁠@HintedNeuron⁠

Instagram ⁠@hintedneuron⁠

Facebook ⁠Hinted Neuron⁠

Leave a review on this podcast at ⁠https://ratethispodcast.com/hintedneuron⁠

Also, subscribe to our email list on our website ⁠https://hintedneuron.com⁠ to stay updated on new releases