126 slides extracted.
Slide 1 — 0:16 (watch)
 | Thank you for joining me today. |
Slide 2 — 0:30 (watch)
 | I'm excited to share some lessons I've learned while leading Claude Code and co-work. First, let me introduce myself. |
Slide 3 — 0:36 (watch)
 | My name is Fiona Fung, and I lead engineering and product for Claude Code and co-work. |
Slide 4 — 0:42 (watch)
 | Before joining Anthropic, I built and led teams at Meta and Microsoft. |
Slide 5 — 0:46 (watch)
 | With that, let's dive into some lessons I am excited to share with you. |
Slide 6 — 0:50 (watch)
 | I hope you find a few helpful tidbits in this presentation. |
Slide 7 — 0:56 (watch)
 | There are five topics I want to cover today. The first is that the bottlenecks have moved. |
Slide 8 — 1:04 (watch)
 | As the bottlenecks have shifted, we need to reconsider our team norms and identify which ones require rewriting to adapt to these changes. |
Slide 9 — 1:08 (watch)
 | I will discuss how we rolled out the updated team norms. |
Slide 10 — 1:22 (watch)
 | In terms of proof, there were several signals that gave me confidence we are trending in the right direction. I hope these insights will be helpful for you as well. Lastly, I encourage you to take this topic back to your team for discussion, as I would love to hear whether any of this resonated with you. |
Slide 11 — 1:36 (watch)
 | With that, let's discuss the first topic: the shift. The bottlenecks have changed. |
Slide 12 — 1:42 (watch)
 | You'll notice the subtitle: "What served you prior may no longer." This is one of my favorite sayings. |
Slide 13 — 1:46 (watch)
 | This growth mindset has been crucial for me, both at Anthropic and in my previous roles at other companies. |
Slide 14 — 1:58 (watch)
 | From all the talks this morning, it's clear that the rate of change is increasing. |
Slide 15 — 2:04 (watch)
 | It is always helpful to evaluate the team norms and processes you have established. Ask yourself if they are still serving their intended purpose. |
Slide 16 — 2:16 (watch)
 | For many years, engineering bandwidth was a costly resource. In software engineering, we focused on protecting this resource. We engaged in extensive planning to ensure that we built the right thing with high confidence. |
Slide 17 — 2:30 (watch)
 | We want to conduct reviews, as there is significant activity when engineering bandwidth is a limiting factor. |
Slide 18 — 2:36 (watch)
 | When considering our industry, it's important to recognize that this is not the first instance where the bottleneck has shifted. |
Slide 19 — 2:42 (watch)
 | Let's travel back to the early 2000s. |
Slide 20 — 2:52 (watch)
 | I had just joined Microsoft and was working at Visual Studio. At that time, there was no such thing as cloud computing. |
Slide 21 — 2:58 (watch)
 | It sounds crazy now, but cloud computing is so ubiquitous today. Back then, however, there was no cloud. |
Slide 22 — 3:06 (watch)
 | I remember how we built Visual Studio in a single server room, where everyone had to be on call for a week. The entire build process operated as a queue. |
Slide 23 — 3:16 (watch)
 | You could only merge six pull requests at a time. Whenever one of the tests failed, you had to debug to determine which of those pull requests caused the failure. |
Slide 24 — 3:22 (watch)
 | Back then, that was a significant bottleneck. However, with cloud technology and continuous builds, that bottleneck has shifted. |
Slide 25 — 3:28 (watch)
 | This represents another shift for us to consider: when coding is no longer the bottleneck, what changes? |
Slide 26 — 3:36 (watch)
 | On the cloud code team, coding is no longer the slow part. |
Slide 27 — 3:42 (watch)
 | That's why we felt the need to change both the upstream and downstream processes. |
Slide 28 — 3:50 (watch)
 | As an example, some of the old bottlenecks included writing code, writing tests, and refactoring. |
Slide 29 — 3:56 (watch)
 | When I first joined Cloud Code, I aimed to onboard and address some bugs. |
Slide 30 — 4:02 (watch)
 | I decided to try test-driven development, which was a significant trend some time ago. |
Slide 31 — 4:20 (watch)
 | The idea is to write the test first, ensure it fails, and then make the necessary changes. This approach provides you with a test from the start. However, it can feel like eating broccoli. Although I actually enjoy broccoli, I use it as an analogy because writing the test first is not always the most enjoyable part of the process. |
Slide 32 — 4:34 (watch)
 | I always felt eager to build a product and experiment with it. With cloud technology, I decided to revisit test-driven development. I thought, let's write the test, but first, it will fail. |
Slide 33 — 4:40 (watch)
 | Let's verify that it fails. This way, I can confirm that the test is actually testing something. |
Slide 34 — 4:52 (watch)
 | I remember making the bug fix, and we already had a test in place. That first experience with test-driven development was much more enjoyable and rewarding. It removed the burden often associated with test-driven development, which was a significant realization for me. |
Slide 35 — 5:02 (watch)
 | Refactoring is another enjoyable topic. |
Slide 36 — 5:06 (watch)
 | I've been on teams where we needed to perform significant refactoring or clean up the software architecture. |
Slide 37 — 5:16 (watch)
 | There is always a debate about when we can find time to do important refactoring or software architecture cleanup. This work often comes at the expense of time spent building the product. However, thanks to the models and cloud technology, refactoring is no longer a bottleneck. |
Slide 38 — 5:26 (watch)
 | Now, when those shifts occur, what new bottlenecks am I beginning to observe? |
Slide 39 — 5:34 (watch)
 | Verification is a significant bottleneck, primarily due to the substantial increase in bandwidth. We need to pay even more attention to ensuring correctness. |
Slide 40 — 5:48 (watch)
 | As roles blur and more people are making changes, it's essential to ensure that everyone feels confident in the correctness of their contributions. The question of who reviews these changes is a significant topic of discussion. |
Slide 41 — 6:00 (watch)
 | How is it maintained? This is interesting because the throughput has significantly increased. We also need to consider the cost of maintenance. |
Slide 42 — 6:10 (watch)
 | These were some of the old processes that we felt had quietly stopped working. |
Slide 43 — 6:16 (watch)
 | I appreciate the phrase "quietly stops working" because it highlights that processes are typically implemented to solve specific problems. |
Slide 44 — 6:24 (watch)
 | We often forget to audit our processes and ask whether they are still necessary or serving their intended purpose. For instance, we had to change our planning norms. |
Slide 45 — 6:42 (watch)
 | When coding is no longer the bottleneck and there is increased coding bandwidth, we had to rethink our planning processes. Code ownership is another interesting aspect to consider. Almost all Cloud code commits are now coauthored by Claude; in the past few months, I haven't seen a commit that wasn't. Code review is also an important topic. |
Slide 46 — 6:54 (watch)
 | I receive many questions about this topic as well. |
Slide 47 — 6:58 (watch)
 | Team makeup is evolving, and roles are becoming less distinct. We need to consider the skill sets that are becoming increasingly important. |
Slide 48 — 7:04 (watch)
 | We are focusing on specific skill sets and emphasizing knowledge sharing. This raises the question of what happens when documentation is no longer your primary source of truth. |
Slide 49 — 7:14 (watch)
 | Given these shifts, we needed to rewrite some of our team norms. |
Slide 50 — 7:22 (watch)
 | We discussed code review, and Kat mentioned Cloud Code Review in her talk this morning. It is an incredible tool that helps us ensure we are making the right decisions. |
Slide 51 — 7:34 (watch)
 | Onboarding has changed significantly. I will share my onboarding story. Planning is essential. When coding is no longer a bottleneck, it shifts our planning, hiring, and organizational structure. |
Slide 52 — 7:44 (watch)
 | Org shape is an interesting topic. We aim to maintain a more agile and flatter organizational structure. |
Slide 53 — 7:50 (watch)
 | Every manager in Cloud Code started as an individual contributor first. |
Slide 54 — 7:54 (watch)
 | With that, how have our planning and technical debates changed? |
Slide 55 — 8:00 (watch)
 | In technical debates, code wins. Building is inexpensive, while arguing is costly. |
Slide 56 — 8:08 (watch)
 | During my onboarding, I wanted to focus on code refactoring. I asked the team what refactoring we considered beneficial but hadn't implemented yet. I recalled having several discussions with Boris about our approach. In the past, I nearly suggested we go to a meeting room and work it out on a whiteboard. However, I realized that thanks to Claude, I could generate multiple solutions instead. |
Slide 57 — 8:26 (watch)
 | I almost suggested to Boris that we go to a meeting room and use the whiteboard to brainstorm. However, I realized that thanks to Claude, I could generate three different versions of the pull requests instead. |
Slide 58 — 8:34 (watch)
 | Boris and I often engaged in productive technical debates. I focused on understanding the impact of our decisions on our colleagues rather than just the implementation details of the code. |
Slide 59 — 8:42 (watch)
 | Prototyping is essential. Whenever we have an idea, we should go ahead and create a prototype. |
Slide 60 — 8:48 (watch)
 | Let's go ahead and use it ourselves, and then we'll try it out. |
Slide 61 — 8:56 (watch)
 | I remember having a prototyping conversation many years ago in another team. There were two camps in the debate. One camp believed that prototyping is great. |
Slide 62 — 9:10 (watch)
 | Prototyping allows for rapid development, enabling you to create a basic framework and gain insights into the product. However, there are concerns associated with this approach. The speed of prototyping can lead to cutting corners, which raises questions about what happens when a prototype appears to work. There's a risk of becoming attached to the prototype, making it difficult to discard the code, and ultimately, you may find yourself trying to ship something that isn't built to scale. |
Slide 63 — 9:22 (watch)
 | It is not built to scale. |
Slide 64 — 9:28 (watch)
 | Prototyping with Claude is an effective way to get started, as it allows us to iterate and learn. |
Slide 65 — 9:32 (watch)
 | Thanks to the model's assistance, we can scale a prototype to production much faster. |
Slide 66 — 9:42 (watch)
 | One thing we reduced on the Cloud Code team is the reliance on extensive design documents and in-depth planning. |
Slide 67 — 9:52 (watch)
 | Most of our discussions occur in pull requests or prototypes. |
Slide 68 — 10:00 (watch)
 | The engineering bandwidth has increased, and coding is no longer a bottleneck. Therefore, we want to focus on verification. |
Slide 69 — 10:08 (watch)
 | I want to ensure that our team continues to improve in verification, especially now that we have increased bandwidth. It's crucial to focus on how we verify quality, which I refer to as "shifting left." |
Slide 70 — 10:22 (watch)
 | For example, it’s frustrating when I hear from our customers and developers that they encountered a bug in the code. I always think, "I wish we had caught it first." |
Slide 71 — 10:34 (watch)
 | What's better than me encountering the bug first is having automation in place to catch it closer to the source. This is something we are constantly focusing on—shifting left and increasing our automation efforts. |
Slide 72 — 10:44 (watch)
 | Who made this change? This was a common question, with many people asking who last modified the code or who the code owner is. Now, I encourage you to dig deeper if you find yourself asking that question. |
Slide 73 — 11:00 (watch)
 | What are you trying to answer? Are you looking for someone who caused a regression or someone to answer a specific question? |
Slide 74 — 11:10 (watch)
 | Are you looking to gain context? Is there a way that Claude could assist you with that? Cat mentioned routines earlier today, and I have a routine set up for myself. Each morning, while enjoying my coffee, I review all the feedback from our various channels. |
Slide 75 — 11:22 (watch)
 | I have a routine that consolidates all the feedback and helps me identify themes. |
Slide 76 — 11:30 (watch)
 | How do you keep up with code reviews? |
Slide 77 — 11:38 (watch)
 | Before we implemented Claude Code for code reviews, I frequently received questions about how to manage them effectively. I can share that Claude Code has been a valuable tool in ensuring we maintain our coding bandwidth. |
Slide 78 — 11:48 (watch)
 | Claude excels at style and linting, as well as identifying obvious bugs. If you have a specification, I encourage you to check it into your codebase, as Claude is effective at verifying against spec drift. |
Slide 79 — 11:58 (watch)
 | It's still crucial to have a human in the loop. |
Slide 80 — 12:06 (watch)
 | In code reviews, human involvement remains crucial, particularly in legal reviews and other areas where risk tolerance is a significant factor. |
Slide 81 — 12:14 (watch)
 | When examining trust boundaries, the principle of "trust but verify" is crucial, as humans contribute essential expertise in this area. |
Slide 82 — 12:26 (watch)
 | Product sense and taste is another interesting aspect. Last December, I wanted to update Claude in our CLI to give it a holiday theme, and I was ambitious. |
Slide 83 — 12:34 (watch)
 | I wanted to turn Claude into a snowman, so I coded up an example and asked our designer to review it. She provided excellent feedback, saying it looked nothing like a snowman and instead resembled Mr. Peanut. |
Slide 84 — 12:58 (watch)
 | I’m not sure if Mr. Peanut is an American brand, but in the US, there's a jar of peanuts with a mascot that looks like a little peanut character. I realized that my initial impression was incorrect; I thought it resembled a snowman, but it actually looks like a peanut. This highlights the importance of keeping human expertise in the loop, especially when it comes to product sense. |
Slide 85 — 13:22 (watch)
 | What should my team makeup be? As roles blur and Claude augments our capabilities, I focus on two key profiles in engineering. The first is creative builders with product sense, and the second is individuals with deep system expertise. This balance is crucial for maintaining trust while ensuring we can verify our processes. |
Slide 86 — 13:34 (watch)
 | I want to discuss product sense. After my talk in San Francisco, many engineers approached me with the question of how to develop this product sense muscle. |
Slide 87 — 13:56 (watch)
 | I want to share what I have found helpful in building product sense, which may resonate with you. For me, it begins with dogfooding. It's crucial, especially for managers, to spend time using the product that you and your team are developing. Whenever I join a new team, one of my first actions is to dogfood the product to ensure I fully understand it. |
Slide 88 — 14:16 (watch)
 | Dogfooding allows you to deeply understand your product. It prompts you to reflect on the problems you aimed to solve and the experiences you wanted to create for users. This practice is particularly beneficial for managers who may not have the time to engage with the codebase. Whenever I joined a new team and participated in dogfooding, team members often expressed appreciation for my involvement, noting that it demonstrated my commitment and allowed me to experience their work firsthand. |
Slide 89 — 14:36 (watch)
 | For me, Product Sense is developed through extensive dogfooding, along with iterating, shipping, and engaging with customers. |
Slide 90 — 14:48 (watch)
 | One of my personal passion projects is working with small businesses. I love small businesses because I believe they are the lifeblood of our community. |
Slide 91 — 14:54 (watch)
 | We recently announced Cloud for Small Business, a project I am very excited about. |
Slide 92 — 15:00 (watch)
 | I have many friends who run restaurants, and they work incredibly hard. |
Slide 93 — 15:08 (watch)
 | I met with many of my friends to onboard them onto co-work. I wanted to understand how small businesses, outside of the enterprise context, engage with our onboarding process. I received valuable feedback from these interactions. |
Slide 94 — 15:20 (watch)
 | It is humbling to recognize the areas in the onboarding flow where we could improve. I encourage you all to dogfood your products; it allows you to truly understand their impact. This experience gives you a deeper sense of how your product performs. |
Slide 95 — 15:32 (watch)
 | If you don't dogfood your products, you may end up making product decisions based solely on metrics, dashboards, or PowerPoints. |
Slide 96 — 15:40 (watch)
 | We emphasize the importance of dogfooding, or as we refer to it at Anthropic, "ant food." |
Slide 97 — 15:50 (watch)
 | Filling cross-functional gaps with Cloud is interesting because I see this across all roles. For example, designers on the Cloud Code Team used to create a red line for any polish or UX fixes and then hand it off to engineering. |
Slide 98 — 16:02 (watch)
 | There is often a wait because engineering prioritizes building products and fixing bugs. Designers wonder when there will be time to implement polish. |
Slide 99 — 16:10 (watch)
 | Designers on Cloud Code utilize the platform to implement many of those polish fixes, which accelerates the iterative loop significantly. |
Slide 100 — 16:34 (watch)
 | It is crucial to prioritize verification. For instance, when I was fixing a bug, I needed assistance with content. As an engineer, I tend to be quite verbose, often providing too much detail when asked to write a concise description of a function. However, the Cloud team served as an excellent content design partner, which I found particularly valuable. The Cloud Code team enhances all roles by supporting areas where individuals may not excel. |
Slide 101 — 16:58 (watch)
 | This slide focuses on management. When I first joined Cloud Code, one of my initial changes was to collaborate with the recruiting team, as I strongly believe in dogfooding. |
Slide 102 — 17:10 (watch)
 | At Cloud Code, we use Cloud to build Cloud, and we utilize Cloud Code to develop Cloud Code. Every manager begins their journey as an individual contributor. |
Slide 103 — 17:34 (watch)
 | I found this approach effective because it allows managers to focus on hands-on coding before taking on the responsibilities of supporting their teams. This time is valuable for managers to roll up their sleeves and engage with the codebase, gaining insight into the engineering experience. Additionally, it's a great opportunity for managers to reclaim some maker hours and dive back into the codebase, as the onboarding process is now much less daunting than it used to be. In my previous roles, even before joining Anthropic, I frequently switched between manager and individual contributor (IC) positions. Each time I transitioned back to an IC role, it significantly enhanced my engineering skills. |
Slide 104 — 17:54 (watch)
 | Onboarding always involved a slight hesitation, as it could be daunting if you hadn't done it in a while. |
Slide 105 — 18:00 (watch)
 | Speaking for myself, I remember when I onboarded to Claude; it was a significantly different experience. |
Slide 106 — 18:08 (watch)
 | In terms of knowledge sharing, the code becomes our new source of truth. |
Slide 107 — 18:14 (watch)
 | When I was onboarding to Claude, I found that the code serves as our source of truth. |
Slide 108 — 18:22 (watch)
 | When I join a new team, I meet with all the engineers to discuss what's on their minds. In the past, I would conduct technical deep dives, but now our conversations focus more on current priorities. |
Slide 109 — 18:34 (watch)
 | I recently conducted my first technical deep dive with Claude, who is excellent at answering my questions. While working on my first bug fix, I asked Claude to teach me about the surface area before I began the fix. |
Slide 110 — 18:46 (watch)
 | What about the areas surrounding that bug as well? |
Slide 111 — 18:54 (watch)
 | For Cloud Code and the co-work team, our code serves as the source of truth. I encourage all teams to consider making their source of truth, whether it's a specification or another document, a skill that is checked into the codebase. |
Slide 112 — 19:08 (watch)
 | Ensure that whatever you establish as your source of truth is maintained in the code base. This approach allows you to keep it up to date. When coding, bandwidth is limited, and this can lead to documentation that is not part of the update loop becoming outdated. |
Slide 113 — 19:24 (watch)
 | To roll out all these changes on the Cloud Code team, it was crucial for us to align as a team on certain key aspects. |
Slide 114 — 19:32 (watch)
 | We also want to ensure we provide space for bottom-up input, as each team focuses on different aspects and may have varying toolchains that resonate more with their specific needs. |
Slide 115 — 19:44 (watch)
 | We aligned on several must-do principles with the teams, which we refer to as our Cloud Code team principles and forcing functions. |
Slide 116 — 19:54 (watch)
 | This slide is outdated. It should reflect that every teammate, not just engineers, uses Cloud Code and collaborates effectively. Using our product daily has been crucial for us. |
Slide 117 — 20:14 (watch)
 | We strive to maximize our efficiency. Whenever we encounter a task, we ask ourselves if Claude can handle it instead. This approach allows us to free up bandwidth for other challenges. Additionally, we give ourselves explicit permission to discontinue outdated processes. Even our team principles and workflows are subject to review. After a few months, we assess whether they still serve their intended purpose, and we are always willing to make changes as needed. |
Slide 118 — 21:10 (watch)
 | There is a lot of freedom for teams or pods to adapt. For example, how Claude integrates into team triage, planning rituals, stand-ups, or which workflows get "claudified" first is largely determined from the bottom up. I’ve found that maintaining a balance between aligning with the team on what’s important for team culture and updating those aspects as needed while allowing the team autonomy is crucial. If I zoom out, the three priorities I have for Cloud Code and co-work that I believe have been effective are: First, keeping the organization agile and as flat as possible, with managers supporting work pods and being directly responsible for parts of the product. Second, our principle of “claudify”: if Claude can do it, Claude should. This is particularly interesting in light of the rapid advancements in model performance. |
Slide 119 — 22:06 (watch)
 | Sometimes, we find that even if Claude wasn't effective at a task two or three months ago, a recent model update can significantly improve performance. This highlights the importance of maintaining a growth mindset and regularly revisiting our approaches, as the models are evolving rapidly. Additionally, team members shouldn't feel the need to conduct all analyses independently, as tasks can accumulate. I recall being on a team with numerous service level agreements (SLAs) for different priorities, such as P0 bugs and incident responses. Eventually, I realized that with so many SLAs, I needed a ranked list to manage them effectively, especially since engineers would express concerns about prioritization given the limited hours in a day. This underscores the necessity of auditing our processes. Now, regarding the proof of our improvements, there are some key metrics I want to share. One significant indicator is the reduction in onboarding ramp-up time, which reflects how quickly engineers can submit their first pull request. |
Slide 120 — 23:02 (watch)
 | The onboarding ramp-up time and the cost to other team members continue to decrease. Often, when a new team member asks me a question, I find myself thinking, "That's a great question; we should ask Claude." Claude has significantly aided our onboarding process and helped answer many questions. This improvement makes it enjoyable for me to return to the codebase. In the past, I often felt guilty about potentially wasting others' time. |
Slide 121 — 23:26 (watch)
 | The PR cycle time should decrease, but it's important to analyze it in detail. Break it down into the various stages involved in a PR process. |
Slide 122 — 23:44 (watch)
 | If your PR cycle time isn't decreasing, it doesn't necessarily indicate a lack of AI adoption. Other parts of your queue may be experiencing delays due to increased throughput. For instance, are your built-in CI systems keeping pace? It's beneficial to break down the PR cycle time into its various components to identify areas for improvement or automation. |
Slide 123 — 24:18 (watch)
 | Lastly, Claude-assisted commits have increased significantly. In recent months, I have not encountered a commit that wasn't assisted by Claude. More importantly than just throughput is the focus on the problems your team is trying to solve. It’s essential to find a way to measure the impact of your efforts, as we aim to make meaningful changes that enhance the product and improve quality. Now, I’d like to share a takeaway exercise for you to discuss with your team. However, I want to be transparent that I still have questions we are working through with the Claude code team. |
Slide 124 — 25:00 (watch)
 | For example, do we still need separate iOS and Android teams? Traditionally, organizations would have distinct teams for each platform, but with Claude, our engineers can work across different mobile platforms. This raises the question of whether we truly need two separate mobile teams. Additionally, how far should we push fully automated reviews? We want to strike the right balance between speed and thoroughness, ensuring we don't overlook anything important. As roles begin to blur, we need to ensure that everyone remains productive. |
Slide 125 — 25:58 (watch)
 | We are currently focusing on verification to ensure that everyone making changes has confidence in their work. It's also important to consider the various parts of the workflow. For instance, I collaborate with my designers, and we recently announced Claude Design. We need to improve the workflows across all functions on the team. I encourage you to identify your noisiest workflow, or any team meeting that you find unproductive or burdensome. I recall being part of a team that held a costly weekly meeting where everyone was on their laptops, only looking up to provide status updates. When you consider the number of people in the room, it becomes clear how expensive that meeting is. Always evaluate whether a workflow is still serving its purpose. If it is costly, consider whether Claude could assist in improving it. Take this process one step at a time. Thank you for listening to my talk; it has been a pleasure. |
Slide 126 — 26:30 (watch)
 | Transcribed by Otter.ai |