tl;dr:“To evaluate whether Copilot should be used org-wide, the authors of this paper conducted an experiment for six weeks, and compared the tool’s impact on a test group versus a control group. They based their evaluation of the tool’s impact using measures for productivity, quality, and security.“
tl;dr:(1) Human memory is complex, with recall activating a network of neurons that can lead to unexpected insights. Stepping away from problems can facilitate innovative solutions. (2) Long-term memory, as opposed to working memory distinguishes experts. Hence, cognitive load becomes a factor. This can be reduced by simplifying tasks or improving information presentation. (3) Experts recognize patterns quickly, while beginners reason line-by-line. Reading more code helps beginners become experts faster. And more.
tl;dr:(1) Human memory is complex, with recall activating a network of neurons that can lead to unexpected insights. Stepping away from problems can facilitate innovative solutions. (2) Long-term memory, as opposed to working memory distinguishes experts. Hence, cognitive load becomes a factor. This can be reduced by simplifying tasks or improving information presentation. (3) Experts recognize patterns quickly, while beginners reason line-by-line. Reading more code helps beginners become experts faster. And more.
tl;dr:In 2020, the code review team at Meta discovered that 85% of developers were satisfied with the code review process in general. They were less satisfied with the speed with which their code was reviewed. This inspired a core hypothesis that the NudgeBot could decrease code review time in 3 ways: (1) The time a diff waits in the ‘needs review’ status. (2) The number of diffs that take over 3 days to close, this timeframe was chosen because they were only nudging diffs after 24 hours. (3) The time to first action.
tl;dr:The report focusses on LLMs e.g. GitHub Copilot and its impact on software development, suggesting it has the potential to improve productivity and reduce cognitive load. However, its benefits are distributed unevenly across users and it introduces new challenges. Key takeaways: (1) Benefits of LLMs in software engineering depend on the specific task e.g. easier to start a project with an LLM but difficult to change generated code. (2) Issues arise with writing prompts and overreliance e.g. burdensome to inspect code, accepting incorrect code. (3) LLMs help the least experienced the most. (4) Adoption is influenced by how well AI tools fit within workflows. (5) Analyzing and integrating information become more important than generating code.
tl;dr:In this issue, Abi outlines the developer productivity metrics used at 17 tech companies, such as Amplitude, Etsy, DoorDash. He then dives deep into several companoes of varying size, notably Google & LinkedIn, Peloton, scaleups and smaller companies. Abi’s advice on how to choose your metrics: start with the problem you want to solve. Is it shipping frictionless, retaining developers by keeping them happy and satisfied, raising the quality of software shipped, or something else? Then work backwards from there.
tl;dr:"Google‘s Engineering Productivity Research team subscribes to the belief that no single metric captures developer productivity. Instead, they break developer productivity down into three dimensions: speed, ease, and quality. Anytime they’re measuring one of the three dimensions (for example, how long it takes for code reviews to be completed), they also measure the other two to surface potential tradeoffs." Quality is broken down into 4 areas - Process, Code, System and Product, which Abi dives into.
tl;dr:What percentage of headcount should be allocated toward centralized productivity teams? Abi found that companies under 1,000 engineers allocate 18.9% of their headcount toward centralized productivity teams, with a range of 8%-37%. The average allocation decreased to 17.8% when including companies with more than 1,000 engineers. Abi breaks this down further by company size and categories of productivity teams.
tl;dr:OKR maturity was found to be positively correlated with the following: (1) Having a unified mission as a team. This is intuitive: a unified mission brings alignment, and this alignment may drive people to set better OKRs as it is clear what they’re working toward. (2) Overall happiness at the company. This aligns with the finding from previous research that progressing towards a meaningful goal is a top motivator for employees. (3) Working remotely. “We hypothesize this is because teams who do not have regular face time and in person meetings with leadership need to be more clear on what they are doing, how it relates to the org, and have better ability to share back their progress.”
tl;dr:Developers were asked to rate a set of challenges to determine which factors had the highest impact. The two most impactful challenges identified were insufficient analysis at the beginning of a task and lack of leadership. Other impactful challenges included missing documentation, demotivation, and information not being made known to the team. Abi discusses how to address these.