2024 Week 24 - Weekly Notes

Reflections

• Used jscodeshift that ChatGPT helped me to create to change defaultProps migration for React 19
• I skimmed through “The Definitive Guide to Google Vertex AI” from Packt Publishing by Jasmeet Bhatia and Kartik Chaudhary. The things I learned about RAG were added to my new blog post
• I met some bird watchers at SJSU this week. Looks like those City Hall Peregrine Falcons are back.

Dev and Tech-y Tech

• Stack Overflow Blog - You should keep a developer’s journal
  • I use my Obsidian Daily Notes. Each of those daily notes are reviewed on a weekly basis (hello Weekly Notes) and may be added to its own Obsidian note. Sometimes they get added as Streams.
• GitHub - Payments 101 for a Developer
• Ben Kuhn - Essays on programming I think about a lot
  • This sounds like something good to write about instead of my lindy library / timeless treasure trove
• Inside Bluesky’s Engineering Culture
  • ScyllaDB | Monstrously Fast + Scalable NoSQL
  • Expo Documentation
• Fortune - How Amazon blew Alexa’s shot to dominate AI, according to employees who worked on it

Recommendations

• Book: Frostbite: How Refrigeration Changed Our Food, Our Planet, and Ourselves. By Nicola Twilly. Amazon

  An engaging and far-reaching exploration of refrigeration, tracing its evolution from scientific mystery to globe-spanning infrastructure, and an essential investigation into how it has remade our entire relationship with food—for better and for worse

• Anime: Frieren: Beyond Journey’s End
• Book: Invitation to a Banquet: The Story of Chinese Food by Fuchsia Dunlop
• Book: Simple Marketing For Smart People: The One Question You Need to Win Customers without Gimmicks, Hype, or Hard Selling. By Billy Broas and Tiago Forte. Amazon
  • I started this book this week
• Book: How to Baby: A No-Advice-Given Guide to Motherhood. By Liana Finck. Amazon
• Blog: Interconnected a blog by Matt Webb

Science

• Quanta Magazine - Most Life on Earth is Dormant, After Pulling an ‘Emergency Brake’
• Locklin on science - Computers reduce efficiency: Case Studies of the Solow Paradox
  • Associated HN thread
• Generalizing to Unseen Domains in Diabetic Retinopathy with Disentangled Representations

Obits

• Françoise Hardy, Moody French Pop Star, Dies at 80
  • Tous les garçons et les filles

Podcast Notes

• Cohere CEO Aidan Gomez sees AI’s pathway to profitability - The Verge
  • Attention is all you need | Proceedings of the 31st International Conference on Neural Information Processing Systems - Aiden is one of the authors
  • On the Dangers of Stochastic Parrots | Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
    • Aiden doesn’t think this danger is bad.

      That’s more than just parroting back what you’ve already seen. I think that these models don’t just parrot back what they’ve seen. I think that they’re able to extrapolate beyond what we’ve shown them, to recognize patterns in the data and apply those patterns to new inputs that they’ve never seen before. Definitively, at this stage, we can say we’re past the stochastic parrot hypothesis.

    • Stochastic Parrots hypothesis

      The claim of that paper is that these [models] are just repeating words back at us, and there isn’t some deeper intelligence. And actually, by repeating things back to us, they will express the bias that the things are trained on.

    • what does stochastic mean in AI?
      • In AI and machine learning, “stochastic” refers to a variable process where the outcome involves some randomness and has some uncertainty. It is a mathematical term closely related to “randomness” and “probabilistic” and can be contrasted to the idea of “deterministic.” Stochastic processes and algorithms make use of randomness during optimization and learning, which allows them to avoid getting stuck and achieve results that deterministic algorithms cannot.
• How AI is eating Finance — with Mike Conover of Brightwave
  • Databricks: Dolly
    • Free Dolly: Introducing the World’s First Open and Commercially Viable Instruction-Tuned LLM - The Databricks Blog
  • AI for the Future of Financial Research | Brightwave
  • Brightwave shared some tips on leveraging LLMs as Judges:
    • 
  • Human vs LLM reviews: while they work with human annotators to create high quality datasets, that data isn’t just used to fine tune models but also as a reference basis for future LLM reviews. Having a set of trusted data to use as calibration helps you trust the LLM judgement even more.
  • Ensemble consistency checking: rather than using an LLM as judge for one output, you use different LLMs to generate a result for the same task, and then use another LLM to highlight where those generations differ. Do the two outputs differ meaningfully? Do they have different beliefs about the implications of something? If there are a lot of discrepancies between generations coming from different models, you then do additional passes to try and resolve them.
  • Entailment verification: for each unique insight that they generate, they take the output and separately ask LLMs to verify factuality of information based on the original sources. In the actual product, user can then highlight any piece of text and ask it to 1) “Tell Me More” 2) “Show Sources”. Since there’s no way to guarantee factuality of 100% of outputs, and humans have good intuition for things that look out of the ordinary, giving the user access to the review tool helps them build trust in it.

    It’s been clear in the last year that the half-life of a model is much shorter than the half-life of a dataset

Other Things

• USA Cricket stuns Pakistan in World Cup T20 upset
  • As someone who’s worked with Indians and Pakistanis, this sounds disruptive
• Food Dive - Chobani founder and CEO buys Anchor Brewing
  • I used to walk by the Anchor Brewing building all of the time. This is refreshing to hear
• Bike Index - Bike registration that works
  • I will want to register my bike on here in case it gets stolen
• The Microsoft Excel superstars throw down in Vegas
• The New York Times - Why 1999 Was Hollywood’s Greatest Year
• Nadia Asparouhova - How to do the jhanas

Notable Videos

Fireship - When your serverless computing bill goes parabolic

Will Larson - How should you adopt LLMs in your product?

Howtown - What your dog sees (w/ Cleo Abram)