The Accidental Gift of Regulation #

In 2018, the European Union passed the General Data Protection Regulation. The headlines were about consent forms and cookie banners. [2] But buried in Article 20 was something more interesting: the right to data portability. Companies of a certain size/scale had to let you download your own data in a machine-readable format.

This was expensive for BigTech. Really expensive. Every major platform had to build export pipelines, format converters, documentation. Facebook alone probably spent tens of millions of engineering hours on compliance. The original intent was to let users switch between competing services, which hasn’t really happened. [3]

But an unintended consequence of this regulation is that while human data analysts don’t typically like writing long, one-time schema definition pipelines and caring about the difference between Postgres’s implementation of timestamps vs. BigQuery’s, AIs will happily do it for you and are really good at [human / machine] language translation.

A Thing That I Did #

Over the past few weeks I’ve been uploading my data exports to Zo Computer, a new tool built by @0thernet and @perceptnet and letting it build queryable databases. [4]

Here’s the datasets about myself that I can now access in a natural language interface:

• Spotify (2012-2025): 180,150 (!) plays across 14 years. Every song, timestamp, device, and skip. [5]
• Amazon (2005-2025): 20 years of purchase history, plus Audible listening, Prime Video watching, and Kindle reading sessions. 1,855 orders total.
• Twitter (2009-2026): Every tweet, like, and follow from the past 17 years.
• Facebook (2009-2025): 16 years of posts, reactions, and event responses.
• Google (YouTube, Chrome, Maps): 30,000+ YouTube videos watched, browser history, map reviews.

Combined, this is probably the most complete record of my digital existence that’s ever been assembled in one place. And I can just ask questions about it.

These models – or perhaps more appropriately said, “their cousins” – are awesome at looking at a cookie trail to understand what I might buy, so why not use them to learn something about myself?

Things I Learned About Myself #

The factoids that emerged were strange and specific in ways no year-in-review summary would ever surface:

• I have listened to 337 days of music on Spotify. Nearly a full year of my life, accumulated over 14 years. 8,087 hours. My pandemic year (2020) alone was 956 hours, almost triple my typical year.
• My most-played song ever is “Go the Distance” from Hercules. 262 complete plays. I value crisp execution and pushing through adversity so it makes sense. Disney soundtracks and musical theater are threaded throughout my top tracks in ways I wouldn’t have predicted.
• I almost never skip Queen. 98% completion rate. Every other artist, I skip sometimes. Freddie Mercury gets my full attention, I guess. Aaaaaayyyyyy-ooooooo
• I’ve spent $79,839 on Amazon over 20 years with a noticeable $/year peak this last year after we moved back to the US. First order: October 7, 2005. That’s not counting the 978 hours of audiobooks I’ve listened to through Audible, which is a separate number that slightly terrifies me.
• My Twitter peak was 2011. 689 tweets that year. Now I post about 80 per year. The arc of my social media engagement is visible in a single query.
• I’ve only ever used the “angry” reaction on Facebook three times. Out of 5,321 total reactions. 93% of my reactions are plain likes. I am, statistically, a mild person online [6]
• [Sensitive findings redacted]. A bunch of things I don’t want to be on the public internet.

A Promise, Long-Delayed but Finally Delivered #

A decade ago, getting these answers would have required writing custom parsers for each platform’s export format, loading them into a database, and writing SQL queries. Maybe 40 hours of work for a motivated, well-caffeinated, technical person. Impossible for everyone else – and frustrating for most data analysts tbqh.

Now I drag a zip file into a chat window, wait maybe 5 mins, and ask questions in English.

The GDPR forced companies to build the export pipelines. They did so grudgingly, and the exports are often messy (Twitter gives you JavaScript files, YouTube gives you HTML, Facebook’s export is pretty incomplete). But the data exists, and language models are remarkably good at parsing weird formats and answering natural language questions about structured data.

This feels like the version of quantified self that was always promised but never delivered. Not just step counts, but your whole digital footprint, queryable.

What Comes Next #

I don’t think most people will do this. Downloading exports is still tedious – the most annoying part is waiting days for the data job to run and catching the email returned. The interfaces aren’t seamless. And maybe a lot of people don’t want to know that they’ve spent 337 days listening to music or $80,000 on Amazon.

But for those who do want to know, the tools now exist. Regulation intended to constrain tech companies’ monopolistic powers accidentally created the substrate for personal data analysis that I think is just the beginning.

The result is that your data, which was always yours in theory, is finally becoming yours in practice. But again, I think this is just the beginning – especially when we consider, for a moment, health.

There is so much structured data in health that is interesting, valuable, sensitive, high-signal, predictive, etc. etc. From ICD-10 diagnostic codes to hormone levels to genetic markers. Of course, it’s high-stakes stuff – for the individual, insurance companies, and doctors – and all subject to HIPAA (in the US), but the fact that the two biggest labs are heavily investing in this space tells me they agree with my diagnosis.

The quantified self, a decade later, finally has the tooling to match the ambition. I still believe it will take us to a better place, newly catalyzed by AI, but let’s leave the doomer-or-pollyanna discussion for another day.

[1] https://karpathy.github.io/2014/08/03/quantifying-productivity/

[2] I still chuckle at https://x.com/Altimor/status/2003019102777172360

[3] If EU regulators understood internet economics, this probably should have been predictable at the time but c'est la vie

[4] There are a bunch of pre-built prompts specific for different BigTech services that Zo lists publicly, but Opus 4.5 is also great at doing ones that don’t have pre-built prompt pipelines if you just talk to it

[5] Thanks Spotify! This was by far the most interesting dataset because (a) music is such a high-baud channel and (b) it goes back to the very beginning of my account’s life. I think this is perhaps because (i) Spotify is a European company so perhaps is pro-regulation and (ii) listening history isn’t very sensitive data if it were to leak

[6] Or want to appear as one (I suspect) :-]

As such, I was learning multiple technical concepts at the same time:

Since then, I’ve collected some wins under my belt as a data scientist. But my best learnings actually came from making mistakes – writing inefficient Hive queries, making low-accuracy or over-engineered models, being overly academic about metric definitions, waiting too long to show business users intermediate progress, etc.

One of the biggest conceptual errors I made starting out in data science was thinking that data analysis was somehow a different, special, disjoint field from software engineering. I’m taking a few hours to write up – and draw up! – my past and current beliefs on this topic.

If you’re busy and only have a few minutes, the main thrust of this post is that data analysis best practices/tools are starting to strongly resemble practices/tools from software engineering and this is a trend I’d expect to continue.

Here’s my experience on this subject represented visually:

If you’d like a more guided tour, please do feel free to read on.

A call to adventure (🎵 here) #

When I started becoming a technical individual contributor, I was scared of “real programming” – which, to me, meant: building features for a web/mobile application, running/maintaining a server, ensuring consistency across multiple database copies,¹ using version control, issuing 😱 pull requests 😱. I was much more comfortable with “data science” or data analysis, which shared a lot of familiar tools like regression with my undergraduate degree in economics.

Now that I’ve worked in data science at multiple organizations in the US/SEA on different analytics problems, when I look back at the concept of data science, I can’t help but feel much like what I imagine Odysseus felt like when returning to Ithaca after battling Sirens and other mythical creatures, or Frodo when returning to The Shire after destroying The One Ring in the fiery chasm of Mount Doom – once you’ve travelled on a long, epic journey, your home never quite looks the same:

That is, I know realize software engineering and data analytics are different heads of the same Hydra, they are inextricably linked – specifically, data analytics is a subset of software engineering! And one that I think will overlap more and more with software engineering in the future, which I’ll now go into more detail to describe.

My past state of the world (circa 2014) #

When I started, I basically knew nothing about either of these subjects, and while it’s always difficult to re-create your old un-informed perspective, as I recall my mental model was something like this:

Software engineering was basically black magic to me – it was writing code that powered websites, mobile apps, etc. – that I barely had any understanding of. In contrast, data analysis, or “data science” as it eventually became branded, was something I could wrap my head around – it seemingly entailed digging into patterns of user behavior, looking for trends, making models to understand causal relationships, etc.²

Whereas the tools of software engineering were “real coding languages” like Java, Javascript, Ruby, I perceived data science to be using “less hard-core ones” like SQL, Python, and R. Now, as someone who’s dabbled in a bit of everything, I realize such hierarchies are quite silly and always extremely reductive!

Further, everything in my comic representation is quite small, because software was just starting to eat the world and data wasn’t yet the new oil. Even though I was excited about, and a practitioner in, data science (and, by extension, software engineering), I still under-estimated how many advances would be made in the fields in just 7 short years.

My current state of the world (2021) #

Here’s how I currently see software engineering and data analysis roughly:

That is, some things (building CRUD mobile web/apps, etc.) are pure software engineering, and some things are pure data analysis (making metric definitions with business users, etc.). However, their intersection is large and growing.

What’s going on at this intersection? #

There’s some really fascinating stuff going on here, let’s dive deep for a second here!

Full-stack ML models #

Most business users don’t want black box models – even if you can show empirically they are 98% accurate. There’s always the chance that your train/test split wasn’t as generalizable as you claim, there could be a structural shift in their industry that a model might not pick up, or they might just be suspicious of ML in general.

Enter full-stack ML models, where a model’s inputs, predictions, explanations, and confidence scores can be presented to the decision-maker in a familiar graphical interface. Even better if you can allow the user to correct mis-predictions in the interface, and feed those back into the model. Tools like Streamlit make it so that machine learning developers can build interfaces without ever leaving Python or worrying too much about deployment.

Metric definitions as code #

When data analytics was just starting, there simply wasn’t enough time to worry too much about aligning a single source of truth for or versioning metric definitions. It was probably common for them to only exist in one place, so why even worry about this? If you want to know what the definition of weekly active users is, you could just look at the underlying SQL in the 2-3 reporting dashboards.

However, now that it’s common to have 50-, 100-, or 500-person analytics teams and million-dollar business decisions made off of dashboards and metrics, we, as an industry, are realizing we need to take this issue a little more seriously.

If Alice the Pricing Data Scientist makes a dashboard that refers to weekly active users and Bob the Search Data Scientist does a deep-dive in a Python notebook on some new feature that also refers to weekly active users, the definition – in code – of those two things should be exactly the same, lest total confusion, frustration, and crises of faith ensue. The situation gets even worse if Alice and Bob were from different departments, who might not even share toolsets. That is, “weekly active users” must become weekly active users.

Large companies have centralized data engineering teams to tackle this over multi-year efforts; for those with smaller data teams, it’s probably better just to go with a pay-per-seat tool like dbt, once you have >10 folks creating and reporting on metrics.

And more #

There’s actually a bunch of other sub-themes in this area I am only starting to explore.

One is automated data quality tests. That is, rather than waiting for a business user to wonder why his team’s KPI inexplicably shot up 30% overnight, when we know 80-90% of the time it’s a data quality issue not a real effect, there will need to be data quality monitoring tools that work across multiple source types, interop with different kinds of databases, etc. I haven’t found an open-source effort or company product doing this well enough yet for my purposes, but am looking!

Data version control is also an interesting concept that my team has started to dip our toes into. No consensus yet, but it seems like a natural tool that will need to be adopted into the machine learning developer’s toolset if she is working on the same model for more than a few months, and doesn’t want to have to look through _v11, _v12 of her past training datasets to find the one that had a good or otherwise memorable result.

Summary #

More and more, software engineering practices are becoming standard data analysis practices. Metric and data versioning are just generalizations of shared code repo practices that SWEs have been using for decades, data quality tests are basically just data-specific unit tests and PagerDuty norms, now long established in SWE ways of working.

I think I’ve realized this because I’ve become less afraid of “crunchy” software engineering problems, and am starting to see that taking a software engineering approach to data analysis yields better and more sustainable results.

Future data tooling landscape (2028) #

The only reliable way to predict the future is to build it, but I’ll hazard some rough guesses of where I see the data tooling landscape going. I expect to continue to see growth in both the software engineering and data analysis fields, but with more software engineering practices being adapted to data analysis workflows.

Or visually:

Everything is bigger, because software and data are just so much more important to every business, every vertical, every industry. But also notice that data analysis has started to ~65% overlap with software engineering!

What might happen at this intersection? #

Let’s zoom in again!

Making ingestion easier? #

Low- or no-code ETL managed services like Fivetran and Stitch will certainly continue to build more integrations, become more performant, and, eventually, lower their prices. In this future, for businesses with no existing modern data warehouse (~75-90% in Southeast Asia if I were to speculate), you could get something basic up and running with just the credentials to their most important SaaS-based data sources (e.g. Salesforce, Google Analytics, etc.).

This is already somewhat possible today, but in my experience, often a company’s most important data sources (e.g. POS system, ERP system, facilities, manufacturing, trading systems) are still either on-premise or don’t yet have pre-built, easy-to-use connectors. So, there is still significant data engineering work to just to get started with modern data analysis techniques. I believe – perhaps wishfully – this time to set-up will go down in the future.

More powerful visualization & exploration tools? #

Like Craigslist’s many sub-marketplaces were bled off into more specific, purpose-built alternatives (e.g. Airbnb for short-term housing, Stubhub for tickets), the Jupyter notebook of 2021 is doing so many core data analysis functions that I expect it will be similarly “unbundled,” though perhaps less dramatically.

The future of the Jupyter notebook? Source

In particular, I find the Jupyter notebook flow of pandas, matplotlib, etc. super clunky for the initial data exploration and cleaning phase of data work. It feels like it hasn’t been meaningfully improved on since I started in the industry 7 years ago. Increasingly, I am starting to rely on more powerful data visualization tools like Apache Superset to do this and, in the future, I expect there will be a proliferation in this field.

Further, if someone is able to finally come up with an interactive, semantic data visualization tool (e.g. “How has the usage of type X users been like since YYYY?”) that actually produces better findings than 3-5h of a good data analyst’s time (e.g. that isn’t just a marketing toy), that product will be worth billions. There are several trying, I’m not aware of any true solutions there, but am keeping my eye out.

ML as a black-box? #

With more and more API-based microservices for the most common machine learning tasks – such as Google Language API for sentiment, entity extraction, translation, Google Vision API for image recognition and classification, OpenAI for a general-purpose language model – I believe two things will happen.³

First, it will be quite easy to get good baseline performance with an out-of-the-box-solution so the bar to build a custom, purpose-built machine learning model will continue to get higher and higher. Second, more work will be focused on stitching together the output of multiple models and make sure data is reliably being fed into them, and implicitly away from the work of developing high-accuracy, performant ones for individual tasks. Note that the first is the classic machine learning development flow and the second is more of a classic data/software engineering task.

Pipelines, pipelines everywhere?! #

Airflow DAGs, cron jobs, Cloud Run jobs, SFTP batch loads, Kafka topics. It’s already a mess today, what will it be like in 7 years? The one guarantee I have is that data engineering tools & approaches that can keep this mess running reliably, transparently, and easily will continue to be the real 10x tool in a data scientist’s toolkit. The most elegant models, visualizations and insights all crumble in the face of bad data.

Thanks #

Thanks so much for reading. I am always looking for more tools to evaluate and for my team to try. If you have a favorite that was missing here, or an area you feel was under-represented or wrongly portrayed, please do reach out over Twitter or email at nicholas.e.huber@gmail.com!

Thanks to Enzo Ampil and Nick Khaw for reading drafts of this.

¹ Though I actually wouldn’t have been able to describe it like this.
² I hadn’t even started learning about A/B testing and the fascinating rabbit hole of product experimentation.
³ I know I said business users don’t want black-box models, but this is about developers, and most good developers are so overwhelmed with requests on their time, they’ll just use the best tool for the job.

More recommended reading: #

• a16z’s modern data infrastructure landscape post
• Matt Turck’s 2020 AI/ML landscape
• Technically’s breakdown of what tools your data team is using
• Jackson Mohsenin’s Interfaces for ML post

Many things that are not (literally) games – love, international relations, negotiations – can actually have their underlying mechanics modelled, and often accurately predicted, within a game-theoretic framework. Game theory is especially helpful to model situations in which individuals’ incentives are not atomic – that is, they interact with one another – which is quite often the case in the real world.

A fundamental building block of the field is a payoff matrix. A payoff matrix is where an individual’s decisions are mapped to “payoffs,” or return values, for an individual taking a certain decision in a moment in the game. For example, if you have $100 at the start of time, and choose to invest it at an interest rate of 10%, at the next time step in the game, you’ll have $110.

These can get a lot more complicated and lead to some really counterintuitive results. One is the prisoner’s dilemma, which can analogously be used to explain (but not justify) how nuclear war may collectively occur between rival nation-states even if individually none of them would prefer this outcome to other alternatives.

Here’s my attempt to partition certain life activities into different kinds of games. That is, the “win-win,” or mutually beneficial, are in the top right and the “lose-lose,” or mutually destructive, are in the bottom left:

Payoff values for life activities in (you, them)-space.

As I reflect back on them, I realize I spent a lot of my college years and early 20s playing “lose-lose,” “win-lose,” or “lose-win” games. As I get older, I’ve become very tired of these and want to play “win-win” games as much as possible.

Related to game theory, I discovered the concept of asymmetric opportunities from Naval Ravikant, the CEO of AngelList, a few years ago. These are opportunities for which there is a chance for a massive upside – building generational wealth, falling in love, shifting the future course of your life – but the maximum downside is small – a small cash expense, a lost evening, a minor ego hit.

Some examples of asymmetric opportunities, according to Ravikant, are: invest in startups, go on many first dates, create a book, podcast, or video, move to a big city, buy Bitcoin, and tweet. Importantly, asymmetric opportunities are nearly always win-win; that is, it is very difficult to get extraordinarily rich without solving an important problem for others at scale.¹

Now that I have grown out of my tumultuous 20s into a stable, happy 30-something, I sometimes look back at some of the asymmetric opportunities that I missed in my early professional life. I write this post as a way to reflect on these, and by way of doing so, potentially share some of my learnings to others.

First, the asymmetric opportunities whose lost upside is pretty easy to quantify and collectively totals around $25 million:

1. I was in contact with Andy Fang, the CTO of DoorDash, in December 2013 when I was a master’s student at Stanford. I had recently discovered the site and it was obvious to me that their high-touch, branded delivery experience was the best user experience in the food ordering space.²

• I was so passionate about it I reached out to Andy through a mutual friend to help them make a demo website for the owner of one of my favorite downtown Palo Alto restaurants, who I knew was just onboarding with DoorDash:

• After I made the site for the restaurant owner, I showed up to his dilapidated apartment at 10pm to demo it.³ He really liked it. But then, he started asking for a lot more features that I didn’t have time to build, and started profusely smoking in the small, confined space. I got a bit weirded out about the whole thing, left, and never talked to Andy or the DoorDash folks again.

• At the time, DoorDash had just raised its seed round, so was probably valued at around $50 million. Today, it’s worth $28 billion. If I had worked for Andy for a few years – and I dunno, made more restaurant websites for him? 🤷 – for 0.05% of the company, I would now have $15 million for my efforts.

2. In 2011, when I was in grad school in the UK, I emailed my friend Nate about Bitcoin and we discussed what an interesting idea it was, and its potential applications:

• If either one of us had invested just $1000 after this conversation in May 2011, it would now be worth $10 million. It didn’t even occur to me to spend 5 minutes researching the technology or figuring out how to purchase some in case it caught on.⁴

Now, the slightly harder to quantify but still large, missed opportunities that, between friends, let’s say add up to roughly another $15 million:

3. When I was an eager-beaver, over-confident undergrad at Harvard Mark Zuckerberg would often come to campus to recruit students to work for him during my freshman year. I remember he always wore these unfashionable chunky black rubber sandals and a preppy North Face vest that never quite seemed to match his overall look. What I do not remember is thinking, for even 5 short minutes, about the interesting social dynamics and connections www.thefacebook.com was enabling, and how that might present me with a worthwhile opportunity.
4. During grad school, I would often read and drink coffee late at night at Philz Coffee in Downtown Palo Alto, where I would have animated conversations with Juan Benet about knowledge graphs and neural networks. Later, Juan would start Protocol Labs, which developed Filecoin and IPFS, cryptocurrency technologies for distributed file storage. I’m still a little hazy on the details of how it works, but as of this writing, Filecoin’s total value, or its market capitalization, is $4.8 billion. Collaborating on something with him didn’t ever seriously cross my mind.
5. When I was an advertising manager at The Crimson in 2006-2008, I directly worked with two guys who were always checking the price of Apple stock on Google Finance with one passionately imploring the other it was still very undervalued. They later, with a third guy, started what appears to be a really fantastic company that is something like an App Store for education, in which students can install and log into different educational apps with a single, central identity. Their company valuation isn’t public but it’s probably worth between $100 and $500 million currently. I spent all of my time at the newspaper focused on hitting my sales quota,⁵ and no time dreaming up any plans with them.
6. In late 2013, I spent 6-10 hours/day teaching myself to program at a Hacker Dojo in Mountain View. I finally plucked up the courage to reach out to the Head of Data Science at Airbnb, impressed him, and made it to a final round interview with his team. Instead of brushing up on intermediate SQL, I spent a lot of time worrying about what to wear.
   • I failed the final round interview on a few difficult questions about [window functions]((https://en.wikipedia.org/wiki/Window_function_(SQL)) and didn’t get the job. I am proud to say that I did turn this story on its head by getting the job 1.5 years later, but of course at a higher Airbnb valuation:⁶

7. Through interning at Facebook in college and later working at Airbnb, I was able to work nearby some data engineering legends. The creator of GraphQL and Dagster was on my summer basketball team and the creator of Airflow and Superset was on my same floor. When I was at Airbnb, I quickly fell in love with Superset’s quick interactive, slice-and-dice features and promise of analytics at the speed of thought. My Superset dashboards had more views than any other data scientist in our 50-person team. Now, both are CEOs at high-potential SaaS startups, borne out of their learnings scaling analytics tasks at these tech giants. I didn’t even once talk to either of these guys for even a 15-minute coffee, or ponder how these technologies could be applied at other companies.

As I reflect back on my college years and early 20s, I can pull out a few key, thematic blind spots in my mindset and behavior that led me to missing out on such quantifiable and unquantifiable opportunities:

• I didn’t have any life foundation. I didn’t have any meaningfully deep relationships or commitments in my personal life. I used to have a libertarian view that such entanglements constricted my freedom and therefore should be avoided. Now, I hold a communitarian point of view, namely it’s impossible for me to know right from wrong, or even formulate basic objectives, without these bonds. My wife, my daughter, my closer relationship with my parents, and my many meaningful work associations give my life the higher sense of duty, purpose, and mission that it previously lacked.
• I didn’t have any explicit goals. My overall life mission prior to ~2015 is 99% captured as “be excellent, get rich.” While this may have helped me in my academic career and establish certain core habits, it was an insufficient life mission. That is, why “be excellent & get rich?” How do you measure it? Obtain it? And for what? What is all the effort for?
• I was often worried about appearances. In college, I was worried about what others were thinking of me, not what interested me. In grad school or my first few jobs, I worried much more about appearing competent than about learning and contributing.
• I was afraid to try new things. I had a fixed mindset and saw new opportunities as threats that I wouldn’t be able to achieve with my current skills, instead of seeing them as opportunities to grow. I overvalued knowledge and perfection and undervalued application. I found safety in the implicit beliefs of those around me rather than having the bravery to form my own.
• I was often only thinking about myself. I was focused almost exclusively on my side of the ‘payoff matrix’ not about the others I was ‘playing with.’ Ironically, I had a large sense of self-importance, but very little true confidence.
• I was focused on the short-term. In school, I spent a lot of time planning my course schedule, but never left time to deeply consider what I wanted to do with the skills and knowledge they contained. I was afraid to make an ambitious, far-reaching goal and not hit it. I frequently focused on insignificant things that could be measured (grades, employer prestige, school prestige, cash salary, appearance, books read), over things that mattered (relationships, love, connection, family, health, wisdom, perspective).
• I didn’t know how to package and sell my skills. I didn’t really know what my unique skills were, nor how to package them. In the years that followed, I would develop a niche and professional reputation in a specific area, which would allow for opportunities to come to me, instead of constantly having to seek them out.

The beautiful thing about having missed out on so much is that means there is much to learn. Failure is always the best teacher, because there is often no compression algorithm for experience. I learned more about sales, management, love, planning, and analytics by failing horribly than I ever did knocking it out of the park on an easy task.

To hammer the point home, one fun counterfactual I enjoy reminding myself about is on time travel. Everyone agrees that if you went back in time, say 15 years to your high school days, to change something, it would have huge ramifications for your future, or potentially the future of the entire planet. However, very few of these same people will admit that they – right now, in this present moment – have the ability to dramatically alter the course of their future life and the future of the world.

Perhaps even more succinctly said:

Or relatedly, what Jeff Bezos would often say when a financial analyst congratulates him on a “good quarter” in 2018:

Those quarterly results were fully baked three years ago. Today I’m working on a quarter that will happen in 2020, not next quarter. Next quarter is done already and it’s probably been done for a couple years…If we have a good quarter it’s because of work we did 3, 4, 5 years ago. It’s not because we did a good job this quarter.

Previously, I would often unproductively and incessantly ruminate about past mistakes. However, my past losses and missed opportunities no longer cripple me. Reflecting on and sharing them gives me a language to explain them (to others and to myself), is an intrinsically rewarding task, and commits and propels me to further change.

It’s difficult to exhaustively map out all of the changes I have made – or am still in the progress of making – that have helped drive the outcomes I want. But in some rough order, these are the things that have helped:

• I have a strong life foundation – my nuclear family, my larger family, my friends and associates – as mentioned before. This community is my reason for achieving, a group to share the spoils and challenges with, and to laugh with and love, regardless of any immediate outcome, positive or negative. They are my one True North.
• I’ve chosen a field to go deep in – analytics/data science, for at least the past 7 years. I’ve developed an expertise and specific knowledge in it, and I am trying to share and grow my knowledge in it more and whenever I can. As the field continues to grow, the value of my knowledge and experience somehow seems to compound.
• I have a running Google Doc on a rough set of goals for myself, in objective and key result format, on what I would like to accomplish in the next 12-18 months. I also have listed down some bigger, more formless goals for 5-10 years in the future.⁷ Goals are specifically important for a few reasons:
  1. They help you make tradeoffs that bring into reality your overall objectives. To give a simple example, one of my priorities is to be comfortable (of course) but I also want to save money for future ventures. It might sound silly but now that I have enumerated the latter and attached specific importance to it, I literally think about this nearly every time I forget to turn off an air conditioner around the house, and the opportunity cost of having done so.
  2. They justify why you need to push through even when it gets tough. If you haven’t explicitly stated what your goal is, it’s incredibly tempting to not see it to completion, either by lowering the bar to make it easier to achieve or just forgetting it entirely.
  3. They give you an objective progress metric to track progress to. One of my 2021 goals was to lose weight, specifically to be down to 100kg by April 1 (achieved on March 15) and 90kg by end December (in progress). By weighing in every week and recording my progress on a simple spreadsheet I am both reminding myself of this goals’ importance through ritual and providing myself with an objective measure of progress for those times if I ever get down about my progress.
  4. The exercise of sitting with yourself and explicitly stating and ranking your goals helps you find your true voice, instead of using others’ value systems to be your definition of success.
• I have tried to adopt a growth mindset to replace my previously fixed one. I have listed out and am working on several experiments that will help me with my growth.⁸ I am more comfortable sharing intermediate progress towards my end goals rather than feeling I need to wait until the end. I am OK to be challenged and try new things.
• I try to carve out specific time for dedicated learning and reflection. I am constantly reading to keep up with the latest trends in my industry and to mentally unwind my brain at night.
• I listen more and talk less. I already know what I think on a subject (writing helps immensely here), what’s more interesting to me is getting others’ points of view. I am confident enough in myself to ask about things I’m genuinely interested in and don’t always need to be the center of a conversation.
• I ask for specific help when needed. If I want to learn from someone else, instead of asking for “an hour to pick their brain,” I’ll ask them what they read to keep up with their area of expertise. If I watch an interview with a founder/business leader that I want advice from or an introduction to, I figure out a way on Twitter/LinkedIn to email them around a specific, clear ask. The response rate, even among senior business/technology leaders, is surprisingly high. Relatedly, instead of being intimidated by others’ success, I am inspired by it.
• I am comfortable to just be myself. First, once you love and accept yourself, you realize life is just too short to anything else. However, over time, you also start to realize that this is also the best strategic option. When you stop trying to be someone you’re not, you can escape competition through authenticity, because no one can compete with you on being you.

Entire industries are being reformed by technology and will continue to be in the next 5-10 years. I’m a diligent, rapid, commercially minded learner, with a little bit of dry powder and a growing tribe, ready to capture the next asymmetric opportunities that come my way. In 2031, I look forward to writing the follow up to this post on the results from further iterations on these newly established core principles.

If you have any thoughts on this, please don’t hesitate to reach out or drop me a note at nicholas.e.huber@gmail.com.

¹ There are certain interesting historical anomalies worth discussing, but I believe they are the exceptions that prove the rule. For instance, colonialism was a massive, once-in-a-generation win-lose opportunity for European settlers and nation-states to the detriment of colonized peoples, but it’s hard to see many modern equivalents.

² As a lazy grad student who couldn’t cook, I was an extremely well informed market participant/connoisseur in the food-delivery landscape. As I recall, in 2013, DoorDash was the only – of at that time many – food delivery platforms that committed to a 45-minute SLA and all meals came in a beautiful, branded DoorDash bag delivered by pleasant (salaried?) deliverypeople. Food from other apps was often late, incorrect and came horribly mangled.

³ I don’t remember why we met so late in the evening.

⁴ Fortunately, I did get a second opportunity in late 2017 after the BTC crash and went in with about 5% of my total assets at the time, which has returned to me ~10x already.

⁵ Which I always hit…but like…who cares now?

⁶ Making my stake comparatively worth less.

⁷ I actually may share these in a future post. Do let me know if it would be interesting. These are always a bit of a work in progress, but, at a high level, I have roughly three big thematic buckets of personal OKRs for 2021: (1) maintain & improve health, (2) learn & earn, and (3) grow & connect.

⁸ Potentially more on this in a future post. Let me know if you’re interested in it!

Compared to working in an office, working from home feels like being liberated from a million nuisances and inconveniences of the day-to-day grind. I have so much more flexibility and control over my time now that I find it difficult to quantify. Since high school, my most productive times have always been late at night, when all is still and quiet in the house, and I’m able to deeply think about things uninterrupted for hours at a time.

I still remember fondly staying up late to write (and re-write) my college application essays, trying to fuse together ordinary public school experiences into a unique tapestry that a distant, inscrutable admissions officer would – through the imperfect medium of 500 words – immediately deem me worthy to enter the halls of power. Later, in college, I remember the beautiful energy of celebrating at 3am when my CS50 final project, some kind of PHP-powered buy-and-sell platform, was finally up and running, and each of my roommates proceeded to post more and more absurd items and prices on it, as it drew deeper into the early morning.

Since entering the working world, I have complied with its typical 9a-6p schedule (often flexing it to 10a-7p like many tech people), but I never really felt I was efficient as in college. And yet I also never felt remote working was an option for me. Somehow, I felt it would be really lonely or distracting, and I didn’t want to be seen as consciously avoiding the office. However, after 2020-2021, when everyone was essentially forced to work from home, I’m surprisingly coming out of it strongly on the side of remote work.

In a work-from-home set-up, I feel a flow state on a day-to-day basis that I haven’t felt since high school or college. I work as a technical manager in a client-facing analytics consultancy/startup. Over the past 12 months, rather than feeling rushed to get to work to get settled, I’ve been able to block my 9a-11a from non-essential meetings, allowing me dedicated time to putter, surf the Internet, read long-form articles, and play with my 18-month-old daughter. In my observation, the quality of my ideas, communication, project management, design documents, technical reviews, proposal writing, presentations, and 1:1s have all gone up.

With my other recent health improvements and the ample focused time WFH has brought, I’ve been able to take on more and stretch myself wider. I’m managing 2-3x the number of projects I was before working from home, I’ve brought in new clients to our firm (instead of just being focused on execution), and I feel I’m being a more involved, present manager to my team. It feels like days are 2x longer than pre-WFH and each day fluidly rolls into the next.

I’ve always struggled with “adult things” – making meals for myself, doing the laundry, showing up places on time; now that I work from home, I don’t have to worry about as many of these real-world nuisances. I no longer have to constantly check my phone to make sure I’m leaving work at the right time to avoid traffic spikes. In a city like Manila, with some of the world’s worst traffic, I’m finally able to schedule back-to-back client meetings, which would often require 1-2 hours of buffer time in between before. I no longer have to worry if my driver will be able to get to the office while my lunch is still warm.²

Because of not having to worry about all this, I’ve been able to deepen my relationship with my wife and my daughter through a thousand little moments and by just physically being there. Instead of cramming some discussion with my wife into iMessage conversations and pictures asynchronously throughout the day, we can just talk about it face-to-face. I am able to observe, celebrate, and contribute to all my daughter’s micro-development milestones.

Within my company and our partner organizations, I’ve noticed that meetings are much more efficient and to the point. Gone are the large, seemingly endless “brainstorming discussions” and “alignment meetings”; it feels like people are finally realizing the opportunity cost of these. Further, when everyone in an organization is working remotely, the incentive for writing and good documentation goes up. I’ve started writing almost anything important down in a public space somewhere, using our Slack channels as pseudo project journals, and am starting to see others do the same and get positive feedback from others on this practice.

There are some downsides to working from home of course. Complex discussions and in-person bonding are much worse over Zoom. One solution to this I’ve heard of from more experienced remote orgs that I’m keen to try is: come together to plan the thing, leave to go do the thing. I think this would be necessary in an entirely remote-first org, where there wasn’t any pre-existing in-person relationship between team members.

I’ve also noticed it can be quite tempting to overwork when working from home, because the line between the two is blurred. I’ve noticed myself sometimes working too much, but simple check-in times (e.g. regrouping with my wife at 10p to watch a murder mystery) mostly solve this. And I’ve always been happiest when my work is a large part of my overall identity, so, on net, I’m used to managing this tension.

I do worry that if someone were struggling (emotionally or with a task) while working from home, they might not know how or, at a minimum, delay asking for help. In an office setting, you have tacit signals to detect this, but when working remotely, it puts the onus on the “struggler” to develop the self-awareness to know when they need to reach out. I don’t mind communicating through the written medium to express frustration or ask for help, so this hasn’t been as much of an issue for me, but I understand this isn’t true of everyone.

I am also fortunate that my wife and I moved into a new condo 3x the size of our previous one in February 2020, about one month before the first lockdowns started in Manila. If I didn’t have a small 9sqm dedicated office space with a sliding door to take calls and have focus time, I do think the work-from-home period would have actually been quite difficult (and quite possibly traumatic) with a toddler running around the house.

Overall, I’m really grateful for the global “forced experiment” of working from home for the past year. I believe it may have accelerated by 10-15 years a fundamental shift in how knowledge workers work and permanently eliminated stigmas associated with remote work. It will be interesting to see if this is possible for me to continue personally and professionally in the future, and to follow how remote work is perceived, compensated, and hired for in 2021 and beyond.

¹ And one very expected source.

² Westerners, please do not overfit on this sentence. My driver is not some white-gloved, liveried servant, these kinds of things are quite common for the middle class in the developing world.

In the conclusion, it implores the reader to “write down” your truly unique qualities – “and don’t edit [them].” That is, ignore the temptation to add positive/negative labels to each, and instead focus on the rock-bottom truth in the features themselves.

Because, what’s important is understanding yourself, so you’re then in a better position to leverage your unique strengths for the benefit of others. The overall goal is to "puzzle fit around you” as the title says, but to do that, in my reading, you first must have a solid understanding of yourself at the metaphorical center.

So, here goes:

1. I can’t stand it when anyone – a boss, a client, a teacher, a call center representative, a lawyer, (previously) a parent, (very occasionally) my wife – tells me what I “have” to do, directly or implicitly. I don’t have to do anything, and can’t stand appeals to authority. I am proudly, fiercely independent.
2. When I am focused on something, I can ignore everything else around me until it’s done – for hours, days, weeks, months. I’ve done this to finish my award-winning dissertation 2 months early, to get a job in data science without any formal training in it, to get into Stanford’s MS in CS without much formal training in it, to win $100k+ business deals, and to write 100k+ view blog posts.
3. I can be extremely impatient. I believe that if something isn’t a priority to do some aspect of it now, it will never get done. I tend to expect this same level of focus out of others, even though they might not share this. Relatedly, I have a terrible short-term memory – I write down all my daily tasks lest I forget them – and my impatience may have developed as a coping mechanism for this. 🤷‍♂️
4. I can jump from topic to topic quickly and easily. I’ve studied with my full passion for months to years of my life the following subjects: law, finance, renewable energy, Javascript, Python, data science, and Final Fantasy Tactics.
5. I am a careful observer. I can often pick up on certain details others ignore (in a slide, presentation, plan, analysis) and generally have a good radar for people’s motivations and unspoken feelings.
6. I need to have a long-term (e.g. one- to five-year) plan. If I feel something I’m doing isn’t progressing towards a goal in that plan, I will quickly drop it. To others, this can appear callous or impulsive, but to me, I merely changed my objective function. Relatedly, I prefer realistic timelines to aspirational ones, and can also sometimes struggle to bring my goals into imperfect reality.
7. I am extremely routine-oriented. My routines ground me, keep me connected to my long-term goals on a day-to-day basis, and when I don’t follow them, I can quickly slip into anxiousness and sadness. I currently practice intermittent fasting, regular afternoon walks & workouts, no deep meetings before 11a, evening debrief time with my wife, and I haven’t drank a drop of alcohol in 3 years.
8. I am self-confident. If someone else can do it, I can do it.
9. I am a precise communicator. I likely can come off as pedantic or inflexible because of this, but I really just want to know what others mean and value the unity clarity creates. Relatedly, I love drinking deeply from good books to see the world through someone else’s perspective and dip into their knowledge.¹
10. I value my time and will spend money to get more time. I value my time between $250-$1000/hour depending on how onerous I find the task, with tax/legal issues at the highest end of that range.
11. I am a chameleon. I can adapt to my surroundings and often even alter my goals and values to match those around me. After moving to Southeast Asia about 5 years ago, I feel more at home here than I ever did in the US. When my daughter was born, I was initially stilted and unsure around her, now I see it as a beautiful, creative outlet and truly unique gift.
12. I am extremely sensitive to taste, smell, and noise. I can detect when our cook has changed my brand of milk, soy sauce, ketchup, or burger supplier. I can smell my daughter’s dirty diapers before anyone else in the household. I wear noise-cancelling headphones for 8-10h per day to focus while working.
13. When I commit, I care and I deliver. I am comfortable to be accountable for important, long-term responsibilities (that I sign up for) and have reaped the benefits of doing this over the past 5 years as I’ve developed a strong professional reputation in my field. When I realized that my daughter is a gift, not a task, I immediately started dreaming up and planning all that we’ll learn and experience together.
14. I generally can’t help but be honest and transparent, even if it’s not the most considerate or optimal thing to do. I struggle with and ruminate over even small white lies that might create points of future misunderstanding.
15. I really care about the people I manage. I had bad experiences with my first few managers and know how painful and damaging it was to have seemingly utterly uncaring, difficult-to-talk-to bosses. At times, my impatience can trump this, but, whenever I have the time, I will take the extra step to mentor and guide those around me.

Oh, and while you’re here, I’ll share one more that I’ve somehow always known and yet only recently re-discovered:

• I love writing. And am pretty good at it. Often, I feel I don’t know what I really think about something, until I write it down, and figure out how to say it in my own voice.² The permanence, intention, and clarity needed in and created by the writing process allows me to crystallize my own swirling, often conflicting thoughts into a single belief, statement, or action.

If you have any comments on this, don’t hesitate to reach out!

¹ Some recent favorites include: The Years of Lyndon Johnson, High Output Management, The Rise and Fall of the Third Reich, Team of Rivals, and How to Hide an Empire. I may do a whole post on this in the future!

² Hence this blog. Thanks for stopping by.