The Challenges and Illusions of Developing AAA Games —— Conclusion

In The Challenges and Illusions of Developing AAA Games —— Part 4 we discussed the failed review, which significantly impacted the project team's morale. The biggest challenge the team now faces is winning back the leaders' confidence in subsequent reviews.

Deciding between going back to the drawing board or making improvements is always a tough choice.

If the project team chooses to start from scratch, they're essentially choosing a path with a high probability of failure (estimated at over 90%). Restarting the project doesn't guarantee better quality, and it leads to a longer development cycle, a further hit to morale, and increased financial costs. The same team and individuals are likely to make similar mistakes in different ways. Good games are polished, not redone, so this direction seems unpromising.

Some teams may opt to change direction entirely. In a competitive environment, success can be seen as breaking away from the past, while failure might be viewed as a lack of persistence. This choice isn't easy to judge as right or wrong; it can only be said to be costly, as much of the previous work would be wasted.

Overall, teams generally choose to refine their versions and continue making improvements to face upcoming challenges.

Let's examine the current status of each team within the project.

Programming Team

Starting with the programming team, they have experienced a significant setback and are now dealing with entirely different issues compared to the initial phase of the project. One of the main concerns is the substantial amount of legacy code accumulated over the past two years. Not all legacy aspects are beneficial, and inheriting someone else's code is often undesirable.

Programmers take pride in their work and tend to view other people's code as inferior. Maintaining legacy code can be a daunting and frustrating task, comparable to continuing to consume unappetizing leftovers. There's a well-known concept in the software development industry called "eating your dog food," which refers to using one's developed products internally to better identify issues. However, maintaining legacy code is often worse than eating one's dog food; in many cases, it feels more like consuming something extremely unpalatable.

"Have you ever come across code that's an absolute mess?" (quote from a lead programmer of a certain game)

During the early stages of a project, programmers who develop demos are typically more experienced and capable of writing high-quality code. Even if they initially create makeshift code to quickly see the results, these seasoned programmers usually possess a strong professional ethic and have more downtime. After all, the project isn't too demanding yet, allowing them to refactor any insufficient parts.

However, the situation tends to change significantly after 2-3 years.

- As the project scales up, the team expands rapidly, bringing in less experienced colleagues who may have weaker control over the code.

- When facing tight deadlines and urgent tasks, there's no time to refactor makeshift codes, causing the code quality to decline. Many people also have bad habits when fixing bugs, such as using various hacks and not writing comments.

- With more people and a larger project scale, everyone's understanding of the program varies, leading to redundant work. Similar functions are duplicated, and some individuals develop something others are unaware of, creating a more error-prone version. Before it can be sorted out, new programmers join, branching out multiple implementations based on these different branches, with code dependencies and function dependencies. Any rushed modification can cause a chain reaction, resulting in a massive program crash.

- Due to frequent changes in functionality, some temporarily unused code is not deleted for fear it may be needed again. Not removing outdated code leads to code bloat, making it harder to find code and increasing the number of modifications. Good development habits can solve this problem.

- As the team grows and tasks become more specialized, there will be more boundary conditions for work. Not every developer has the ability or willingness to take on additional boundary work. Coupled with communication difficulties, everyone tends to protect their territory, causing program functions to accumulate at individual interfaces, forming a moat of redundant functions that isolate errors, reduce efficiency, and harm the project.

- AAA games may have multiple platforms and versions, leading to a large number of macro definitions, which can be frustrating. The cost of reading code increases, especially for extensive macroblocks with various nested structures. These complexities can be so overwhelming that even Integrated Development Environments (IDEs) struggle to accurately distinguish valid and invalid macros in the current build. Many tools attempt to analyze macro situations accurately, but they either generate errors or are extremely slow, taking away the joy of programming.

These situations occur to varying extents in different teams, and production efficiency generally declines in proportion to the project's level of disarray.

Is refactoring the solution to these problems? It's not that simple.

The code is riddled with issues, making it challenging to refactor and turn things around. Leaders are focused on progress, and basic unit tests are incomplete, so refactoring carries risks and offers no short-term benefits. Long-term benefits may not be apparent either; it's difficult to discuss long-term advantages with a programmer who has been working overtime for weeks, staying up late and only wants to quit after receiving their year-end bonus.

Furthermore, many teams use the term "refactoring" to justify various shortcuts, hoping that refactoring will help sort things out while effectively prolonging the task's development cycle. Planners and artists may feel envious, as they don't have professional terms in their fields that sound as prestigious as "refactoring," which seemingly allows for delays while still being politically correct. The closest term is "redo," but it pales in comparison to "refactoring." At least refactoring is often done when there's less pressure and no need for overtime while redoing typically occurs under immense pressure, with all-nighters and sleeping at the office being common.

Art Team

Although they are relatively independent and not as affected by programming issues, they face their unique challenges with art resources.

- Art resources can vary in quality, requiring sorting and conversion. While this workload is manageable thanks to various tools and the option to outsource, it can be overwhelming when dealing with a large volume of art resources in a nearly finished game.

- "Darn it, the programmers added new rendering features to impress the boss again." Since art assets rely on the game engine, they can be unstable. A small change in the engine might cause the art architecture to collapse and need rebuilding. Programmers complete their work, enjoy the technology, and receive praise. However, since energy is always conserved, if programmers have more positive energy, the hardworking artists need to balance it with some negative energy. Their joy is my disaster. Artists need to adjust various features in the levels, and if not done well, the boss won't blame the programmers but will look for the art team members. The most frustrating part is that programmers often implement imperfect technologies with several flaws. Whether due to irresponsibility or genuine technical limitations, the boss only sees the positive aspects. As for those flaws, programmers casually mention that artists should be cautious to avoid issues when working on them.

Artists can face various challenges due to the relatively low cost of changing art assets. These challenges are connected to both the artists themselves and project management. Let's discuss:

1. Low wages for art developers: Artists in the gaming industry often face lower wages compared to other roles.

2. Intense competition in the art field: Art is an ever-evolving field with fierce competition. Art developers generally have a higher tolerance for pressure. Unlike programming, which often involves collaboration and synergy in project development, artists usually work independently and can be easily replaced. This is similar to the saying, "A single chopstick is easy to break, but a bundle of chopsticks is not."

3. Project management challenges: Ideally, everyone should be working on something at all times. For programmers, particularly those in rendering and engine development, root-level changes can impact various aspects like game design, toolchains, and art. Therefore, it's preferred to consolidate during the later stages of a project. However, artists are at the "leaf node" of a project. When a project isn't going smoothly, artists are often asked to modify the UI to create the illusion of continuous hard work. Moreover, UI design is subjective, so projects often involve constant UI changes, sometimes even complete overhauls. It's important to note that no project has succeeded solely due to its UI, so unnecessary changes should be avoided.

4. Workload fluctuations: At the beginning of each version update, artists are the busiest, even if there's no actual work to be done. However, towards the end of the update, most artists may have some free time to chat, joke around, and tease planners and programmers, which can be a fun aspect of the job.

Planning Team

Finally, let's discuss the heart of the game - the planning team.

In the beginning, planners were filled with enthusiasm and ambition. Now, it's time for them to bravely step forward and take responsibility for any issues that arise.

"All problems are the planners' problems, even the crashes," (By players who are unaware of the truth)

Planners can also feel unfairly blamed.

Diligent planners might say: The promised features were not completed by the programmers until just before the launch, so how could I have had time to fine-tune the gameplay?

On the other hand, lazy planners might secretly hide requirements and only inform the programmers at the last minute, passing the responsibility onto them and then complaining like the diligent planners. Wherever there are people, there can be procrastination. Bureaucratic systems can systematically delay task progress and evade responsibility, which is an art in itself. The highest level of this skill is to leave no trace or evidence. Always ask for instructions, avoid taking responsibility, and prioritize reducing workload. Whether the game is enjoyable or not is not their concern, as long as no problems arise on their part in the project.

The planning team faces the dilemma of whether or not to make significant changes to the gameplay. If they change it, it might not necessarily be more enjoyable; if they don't, it might not necessarily be well-adjusted. However, they can only choose one path, and the unchosen path is reserved for the leaders: when the change fails, the leader will say, "Why didn't you choose the other path?" It is clear that strategies are never wrong; it is always the execution that is at fault.

Another major challenge for planners is the delay in feedback. For any job, fast feedback makes the work more enjoyable and easier to improve. For example, rendering and engine efficiency optimization have relatively easy-to-set standards, with clear distinctions between good and bad results, and quick feedback. However, verifying gameplay design by planners is challenging, as it requires longer discussions, implementation by programmers, and reluctant changes by artists and programmers. After adjustments, playtesters are needed for the experience, making the optimization process time-consuming. Trust between teams is crucial; if trust is lost, programmers and artists may be unwilling to accommodate, making the planner's job even more difficult. When a project review fails, planners often face the most significant doubts, leading to a crisis of trust and making work more challenging to carry out.

Each project team is different, with varying issues in different aspects, so it's impossible to generalize. The weakest link will seem to have the most problems. If planning is the weak link, it will feel like a good planner would make everything go smoothly. However, once a good planner is found, programming might become the new weak link. The core issue here is that the quality requirements for games are incredibly high, and there is always room for improvement regardless of the current level. Planners, programmers, and artists can only strive to move forward, not aiming to be the best but to avoid being the slowest among the three teams.

Unfortunately, people are not machines, and even machines get tired, let alone humans. Repeated setbacks will prompt the team to reevaluate their situation: Should we continue? Is the producer competent? Is this team not up to par? Should we change jobs after receiving the annual bonus?

"People's hearts become divided, and leading the team becomes challenging." 

The project's outcome is no longer the top priority; everyone starts preparing for their future. In the past, calls from recruiters were quickly dismissed; now, perhaps, they engage in conversation, accidentally forming a connection, and moving on to a new job for another project.

Motivation is useless when people are exhausted and have lost their passion for the project. Once enthusiastic team members, after facing constant setbacks, lose their drive. Proactive and dedicated individuals will bounce back from failure, striving to learn and improve, hoping to do better next time. In contrast, passive and unmotivated individuals will do the bare minimum, refusing to work extra hours. Even if they are physically present, their hearts are not. By the way, they may jokingly claim to have a brave heart and shout "Freedom!" even when faced with performance evaluations.

The cycle of project reviews and disappointments continues. Looking forward, the project's beginning was full of excitement; looking back, the project's efforts eventually returned to a state of calm. The heat death of the universe is an inevitable fate for all things.

The rise was as swift as slicing through bamboo, but the decline came like a mountain collapsing. When the wall falls, everyone pushes; when the tree falls, the monkeys scatter. The project consumed your youth and dreams, leaving nothing but regrets.

Years later, no one will remember the impressive demo you once created. It was a part of your past but did not shape your future. That demo video, along with your presentation slides, lies forgotten in a corner of your hard drive.

Conclusion

Projects that drag on for too long can result in being unfinished, utilizing outdated technology, having worn-out teams, and facing shifting market demands.

Similarly, overly lengthy articles may also remain incomplete, with dwindling interest, excessive workload, and a lack of inspiration.

I have been involved in several large-scale projects, including two games that each took around four years (fortunately, I joined later in the process) and another game that took five years. Thankfully, all of these projects were successfully launched, so you can trust my work experience. However, even in successful projects, many issues resemble those encountered in failed projects, making it worthwhile to discuss them collectively.

Many people wonder, with so many issues mentioned, where are the solutions?

In the article, there are no perfect solutions presented. Software engineering is a field that continues to evolve rapidly and is a complex science, making it difficult to pinpoint a precise method for creating quality software. Game software engineering has its unique challenges, which cannot be solved simply by working overtime. Additionally, bringing in various senior consultants often leads to miscommunication and may not effectively guide the work. This discussion also takes into account human factors and market forces, which are beyond our control.

However, I believe that if this series of articles can spark some reflection and introspection among readers through its humorous approach, then its purpose has been fulfilled.

Successful projects often involve a mix of various opportunities and coincidences, and even with numerous dramatic elements, they can still be turned around under wise leadership. Countless unique cases cannot be replicated, and the advice given by enthusiastic sharers may not be directly applicable to your projects. Additionally, their sharing might be biased, as they could be embellishing their decisions and exaggerating the challenges they faced. As a result, it is hard to gain valuable insights from successful projects.

On the other hand, studying failed projects is a more worthwhile approach. Learning how to avoid problems and being aware of potential pitfalls beforehand can help with preparation. There are always numerous pitfalls, and even if you know where they are, you might still fall into them. Focusing on avoiding these pitfalls makes the process a bit easier.

As for how to make a project successful, I cannot provide a definitive answer. To those who have asked, I would like to pose a counter-question: If I knew how to make a project 100% successful, wouldn't I be busy starting a business with others instead of writing articles here? Theoretically, strategic planning is always sound, and the only mistakes lie in execution. What we need to do is not to focus on ensuring success from a distance, but to be practical and think about how not to fall into pitfalls. At our current level, "I have a solution" is far more valuable than "I have a dream."