Every so often, we find the most interesting data science links from around the web and collect them in Data Science Briefings, the DataMiningApps newsletter. Subscribe now for free if you want to be the first to get up to speed on interesting resources.
- 2022: A Year Full of Amazing AI papers- A Review
A curated list of the latest breakthroughs in AI by release date with a clear video explanation, link to a more in-depth article, and code. - Prompt Engineering Guide
“This guide contains a non-exhaustive set of learning guides and tools about prompt engineering. It includes several materials, guides, examples, papers, and much more. The repo is intented to be used as a research and educational reference for practitioners and developers.” - How does GPT Obtain its Ability?
Tracing Emergent Abilities of Language Models to their Sources - Forward-Forward Algorithm App
This app implements a complete open-source version of Geoffrey Hinton’s Forward Forward Algorithm, an alternative approach to backpropagation. - AI’s Jurassic Park Moment
“Something incredible is happening in AI right now, and it’s not entirely to the good.” - Amazon’s AutoML vs open source statistical methods
TL;DR: We paid USD $800 USD and spend 4 hours in the AWS Forecast console so you don’t have to. - We Evaluated ChatGPT vs. Google on 500 Search Queries
“We measured ChatGPT vs. Google, and found that ChatGPT crushes Google on coding queries and ties it on general informational queries — despite not being optimized for a search experience at all. Dive into this post to learn more about OpenAI’s existential threat to Google.” - The GPT-3 Architecture, on a Napkin
“There are so many brilliant posts on GPT-3, demonstrating what it can do, pondering its consequences, vizualizing how it works. With all these out there, it still took a crawl through several papers and blogs before I was confident that I had grasped the architecture. So the goal for this page is humble, but simple: help others build an as detailed as possible understanding of the GPT-3 architecture.” - Thinking Like Transformers
“Thinking like Transformers proposes a computational framework for Transformer-like calculations. The framework uses discrete computation to simulate Transformer computations. The resulting language RASP is a programming language where every program compiles down to a specific Transformer.” - Machine Learning at Monzo in 2022
“We’re no longer a machine learning team in a company – machine learning is an established tool that is used by many disciplines to help teams reach their goals.” - Building a synth with ChatGPT
“I know everyone is talking about ChatGPT right now. One aspect I haven’t seen much of is what it actually looks like to build software using it.” - Failed ML Project – How bad is the real estate market getting?
“There aren’t enough failed data science projects out there. Usually, projects only show up in public if they work. I think that’s a shame.” - TikTok’s Secret Sauce
“TikTok’s algorithm is ordinary. Its real innovation is something else.” - Why Business Data Science Irritates Me
“Scientists come in wanting to build cutting edge models, and most of the time we have to crush their dreams, and tell them to start by solving the problem with SQL, and if absolutely necessary, a linear regression.” - Pretraining Without Attention (paper)
“Transformers have been essential to pretraining success in NLP. Other architectures have been used, but require attention layers to match benchmark accuracy. This work explores pretraining without attention.” - Learning Interpretability Tool
The Learning Interpretability Tool (LIT) is an open-source platform for visualization and understanding of NLP models. - tiktoken
tiktoken is a fast BPE tokeniser for use with OpenAI’s models. - OLM RoBERTa/BERT October 2022
“This is a more up-to-date version of the original BERT and original RoBERTa. In addition to being more up-to-date, it also tends to perform better than the original BERT on standard benchmarks.” - I used the #StableDiffusion 2 Depth Guided model to create architecture photos from dollhouse furniture
By using a depth-map you can create images with incredible spatial consistency without using any of the original RGB image.
