Q1 2023: My Last 3 Months of Exploration

One year ago, I departed from Uber to cofound a startup. 3 months ago I left my role of CTO from my startup. Since then, many friends have reached out to me asking what is next, and would like to keep in touch. While I would love to chat with everyone every so often, I quickly realized that it would not be scalable. :)

So I decided to use Substack as a way to share my latest thoughts and learnings on tech topics like big data, AI, blockchain, as well as non-tech topics like business and lifestyles. If you want to stay in touch, please subscribe.

In this first post, I would like to share my recent learnings as well as some unsolicited advices that might be useful to my friends.

1. What am I up to?

In ML, there is a well-known trade-off between “exploit” and “explore”. After working in various tech companies for 17 years and a short stint as a cofounder of a startup, which I would call “exploit” of my prior knowledge, I realize that it’s time for me to “explore” more.

In Q1 2023, I took the following explorations:

• Deep Learning: On Coursera, I completed all 5 courses in Deep Learning Specialization, and audited some of the courses in Natural Language Processing Specialization and Generative Adversarial Networks (GAN) Specialization. For my tech friends interested in AI, I would highly recommend that you spend ~100 quality hours on these. The foundational knowledge is crucial, and it’s practically hard to get back to these foundations once we get busy building products and solving customers pain points.

• Reinforcement Learning: I spent ~50 hours reading Reinforcement Learning: a brief introduction. The Reinforcement Learning Specialization on Coursera is also great, although I felt I could learn more efficiently with the book.

• Centralized Exchanges and DEFI: I used to code against the trading APIs on various centralized exchanges likes Coinbase Pro in my spare time, however, I never got sufficient time to learn about smart contracts, decentralized exchanges, and DEFI protocols. I spent ~100 hours in writing code as well as learning about ERC-20, Uniswap, 0x, and related. ChatGPT has helped me to learn and practice a lot faster.

• Listening to Audible.com books: Here are a few of my favorites:

  • Genius Makers: A colorful history about the AI advancement in the last 15 years, and the stories among key AI leaders including the Nobel Laureates, the innovative startups like OpenAI an DeepMind, and the big tech companies. I bet you will love this book!

  • How to Create a Mind: An interesting story about human’s hierarchical mind, and the difference between AI and human brains.

  • A Thousand Brains: The difference in the structure of human neocortex and the deep neural networks.

  • How Emotions Are Made: A debate on whether emotions are biological or cultural.

  • Social: A theory that social skills are special and different from other skills of human beings. Social skills helped human being as a species to succeed, and will continue to do so.

2. What is next?

I very much enjoyed the 3 months of exploration, and I plan to continue for the foreseeable future, potentially working on multiple ideas at the same time before committing full-time to a single idea (when the exploit phase starts).

• Evaluate some big problems to solve:

  • Use AI to help software and data engineers: To solve a problem well, one needs to have a deeper understanding of the problem first. Software engineers and data engineers, including myself, know the pain of our own jobs the best. This is an interesting article about the landscape of AI-assisted developer tools. It can be considered as a good starting point.

  • Meta Learning: Let’s assume that AGI stands for the level of AI that is equally intelligent as human beings. I would argue that AGI needs to be able to invent AGI itself, assuming that at some point, human has invented AGI. This is because otherwise, AGI is still inferior to human intelligence, since after all, human is able to invent AGI, but AGI itself cannot. I treat these concepts interchangeably: the intelligence that is able to create intelligence, the ability to learn how to learn, the Master Algorithm, and Meta Learning.

  • AI for Science: The IMO Grand challenge is very interesting to me. I was very much into science competitions in the early half of my life, from physics to mathematical modeling to programming (IOI 1999, Google Code Jam 2004), and had met two of the nowadays IMO Grand Challenge committee members. I would very much want to join that journey although I wonder if I would resonate with the emotional feeling of the famous Go players once AI is able to beat humans in those competitions.

• Making some short-term impact:

  • Use AI to improve machine efficiency: A large part of my career was in writing large-scale and efficient software, as well as optimizing software to run more efficiently. These can have immediate impact to save energy and hardware, and are bigger and bigger problems with the data center and cloud infrastructure spend.

  • Use AI to help improve liquidity and market making: High liquidity is important for the health and efficiency of a market, no matter what market it is. I had some early wins in applying ML in investing in Lending Club Notes as well as trading on centralized exchanges. Search engines, realtime ads bidding, online ML learning systems, and automatic trading bots have very similar technical challenges. They can all benefit from better infrastructure and better AI.

  • Practicing in some Kaggle competitions: I still haven’t spent enough time to practice what I had learned in DL and RL. These competitions could be a good starting point. It will allow me to get into the user community of open-source ML and AI frameworks like PyTorch, Fast.AI, Ray, etc. Let me know if you would like to team up!

2. Unsolicited advice for my friends

In the last several weeks, the media is overwhelmed by the latest launches of ChatGPT, GPT4, and others. Almost every day there are something new, and it takes a lot of time just to keep up with what’s happening. This is especially true for my friends who have a full-time job. Many of them are busy, curious, and also anxious.

My advice is simple: Focus on how you and your work will be impacted by these new technology. If you are a software engineer, try out Copilot. If you are a designer, try out Midjourney. If you are learning something and have questions, try asking ChatGPT.

If you have a lot of free time to explore, you could try a lot of new AI products and maybe read a lot of papers. But if you have a full-time job with exciting problems to solve, don’t let the media buzz distract you from doing real impactful work at hand. After all, AI is just another tool to help you achieve your existing goals easier and better; it doesn’t and shouldn’t change your goals.

3. If you would like to see my updates, please subscribe to my Substack!

I would love to hear from you on what you are working on, as well as any feedback, suggestions, and pointers for me! Please comment here or send private messages on LinkedIn.

---