We have Moved!

Massively Multi-Player Game Development blog has moved off of the HardcodedGames.Com site to it's new home here on Blogspot.

Show Full Story

Hard Coded Blog: Nintendo Domination!

Video Game analyst Todd Mitchell from Kaufman Bros. has posted some interesting analysis that we rant about on the Hard Coded Blog.

Show Full Story

Hard Coded Blog: Microsoft is Dead...is EA next?

There is a new mini-rant on our sister site: Hard Coded Blog asking if Microsoft and EA have lost their Gorilla status.

Show Full Story

Large Scale Project Management

This is an excerpt from the Article

Large Scale Project Management
by Michael Saladino






Projects across the industry are becoming more complex and in few genres is this more evident than in massively multiplayer games. Publishers are pushing for a shorter turn-around on their investments, hardware complexity is increasing and end users are demanding larger and more interactive content. All of this results in ever-growing development teams. At major studios like Electronic Arts, Sony, and Microsoft, teams over a hundred people are becoming the status quo. This gem will look at what the new project management looks like in terms of team hierarchy, rigorous scheduling, strong process, and resource handling. I will propose a collection of guidelines that I have used myself in directing large scale projects at Microsoft and now at Electronic Arts.

Building Your Hierarchy

For decades, computer games were developed by small teams with a few dozen people at the most. This helped promote the idea that a couple of managers could deliver a game in its entirety. One producer, one lead engineer, one art director and two dozen artists and programmers building the game was how most products were being constructed. But with larger, modern teams comes the necessity for hierarchy to maintain direction and order. Deciding on your hierarchy or organization chart is the first step in building a stable and productive development team. But what does hierarchy really mean for a large scale project?

First and foremost, hierarchy is a way of reducing the number of direct reports to a single manager. This is normally done by selecting sub-leads below the manager. If you have one lead engineer and twenty programmers all reporting to him, you need to select sub-leads in order reduce the communication load on the manager. Pick four or five engineers along major functionality lines such as multiplayer, AI, game mechanics, front-end, graphics, and systems. The goal should be for each sub-lead to have two to four reports and those four or five sub-leads all report to the lead engineer. This is the most fundamental purpose of hierarchy: reducing communication requirements by empowering sub-leads to take ownership.

By reducing the number of direct reports one manager has to deal with, you are clearly reducing his daily and hourly communication needs. Studies in the military, arguably one of the greatest achievements in management structure, find that an individual manager functions best when he’s only handling three or four people directly. From my experience in game development, I find that someone can effectively produce assets for a game whether art or code and maintain three or four direct reports. Once you hit six to eight, the manager’s ability to produce drops extremely low as they spend their days managing almost exclusively. And if you cross over into double digit reports, an individual’s ability to even manage effectively is affected. Therefore, keep direct reports in the single digits. If the manager is also expected to produce assets for the game, keep them below five; otherwise, they are a pure manager.

While the direct result of hierarchy is improved communication handling, there are other indirect and subtle improvements to take note. Workers in small groups tend to feel more individual empowerment. When a person is in a cube farm and is treated just like the other thirty engineers surrounding him, the person can begin to lose identity. A feeling like an assembly line worker can take hold which is usually detrimental to the person’s overall production. Creating sub-leads in control of small groups can keep a person feeling like a major contributor even if it’s only in the small group. Most engineers would rather identify as one of three engineers building AI rather than one of thirty engineers build a game. It’s simply more personal.

Of course, this benefit also has a draw back which is people becoming isolated from the game as a whole. If an artist does nothing but paint household object textures for months on end, it can be easy to lose the forest for the trees. The detail that they’re focused on grows in size until it becomes the whole world which can be good in the short term but is often bad for the long. Remember to always keep people attached to the overall product. Have weekly team meetings where you show off the game as an end-user would experience it so every person can look and say “I did that part”. If you think about this issue in a psychological way, people in the game industry were most likely all smart children in their school days. Therefore, take them back to the joy of seeing their work posted on a board with a gold star for everyone to see. Even if the grown-up cynic in them openly scoffs, a part of their childhood will appreciate the attention.

Delivery vs. Functional

Once your team grows to the point where more hierarchy is required, the question becomes how do I divide the team? If you imagine a small start-up with six engineers and twenty artists most people would split the team up into functional groups. Five of the engineers would report to one lead. The art team would be cut up into modelers, texture painters, musicians, and animators each with a corresponding lead. And this layout would work well as I’ve experienced it many times in my past. But when you’re game requires ten times this many people, does a functionally based organization still make sense?

Do all the administrative assistants at IBM report to a single lead administrative assistant? Do all the engineers at Microsoft report to a single lead engineer? Can you imagine what would happen if they did even with a deep sub-lead hierarchy like I described above? It would be abject chaos because a functional hierarchy is useful at handling functionally dependant communication but little else. This causes a hierarchy based on functional lines to not scale well. Our true tasks in constructing a game are not functionally pure but instead require constant cross-functional communication. So let’s try something different: a hierarchy based on delivery instead of department.

A delivery group for a game would be the front-end team. This group would definitely include engineers and Photoshop artists. It might also include usability experts, 3d modelers, and Flash scripters. We now have many different disciplines all reporting to a single front-end manager with possible functional sub-leads beneath like a lead front-end engineer. The team is given a single mandate from the team leader: deliver the front-end needed to ship the game. The front-end manager would drive all external information gathering such as the front-end design documents and technical certification requirements from 3rd party console. But the majority of information needed to build this unit has been isolated within the walls of the front-end team and only team wide critical information ever needs to be floated out of the group.

Other delivery groups could include game mechanics, systems, multiplayer, or a stand-alone server. These are areas that can be sliced from the whole and driven with minimal external communication because a delivery group is cross-functional by nature. Included in the multiplayer group are the levels teams that will construct the multiplayer levels. There are modelers, designers, and texture painters. We include the engineers that will build the client/server architecture and extend the single-player experience to include multiplayer. By making these divisions through the overall team, you can reduce the problem from 160 people making one product to four teams of 40 making four products. And if you draw the lines cleanly and have a strong management team at the top of those four products, you’ll be able to keep a rolling integration of these components during development.

Pure Art Directors

But delivery groups aren’t the best solution for all problems. While they keep a smaller team focused on a subset problem, you do wind up with an unfortunate long-term issue with people having managers that aren’t of their discipline. For instance, imagine an animator being assigned to a group that’s going to deliver the melee system for a role-playing game. His manager is handling different disciplines and may not really understand an animation skill set or career. How will this relate to the animator’s ability to get promotions? Perhaps the animator is significantly better than how the melee system turns out due to engineering issues. How will the animator’s pure talent be seen unless a more senior animator is somehow in the equation to rate his long-term value to the company?

This is why functional leadership is needed. Notice I did not say management because these are two very different beasts. To manage as I’ve said before is to handle information flow. Leadership is to inspire your team and motivate them to become better than they currently are. An art director or even more specifically an animation director for my above example needs to be in place to help drive quality and identify long-term career growth for the functional team. No one reports to the art director, technical director, or game design director because they are the holders of the pure art. They push the quality limits without having their hands in the schedule or management flow. So by including these functional leaders in the team you allow managers to drive delivery teams while allowing pure art owners to help inspire individuals with a functional language that they share.

Producer vs. Development Manager

The next line we need to identify comes in the form of the producer. Many small companies that I’ve worked with over the years have a producer come up with ideas, schedule them, and then push the team to maintain vision and deadlines. What a schizophrenic position. To be expected to both develop brilliant new ideas and have the resolve to cut them when needed. No wonder so many titles scream madly past their delivery dates. It’s like allowing an author to edit their own book or giving a director complete control over the final cut. Maybe there are a few true geniuses where this is a good idea but for the rest of the human race, it’s a recipe for disaster. The idea man will always want the final result to be grander in its ultimate pursuit for perfection and you want him thinking that way. You need another person to be the scheduler, manager, and all-around “get it done” man. Enter the development manager.

A development manager is a producer’s alter ego. When the producer is blue skying, the development manager is there to attach the anchor of reality. The development manager’s job is to maintain the schedule and manage the time of his reports. A successful development manager seldom gives the producer exactly what he wants but instead masters the art of delivering 90% of what the producer wants but at half the cost. Each delivery group should have a development manager at the helm making sure that milestones are being met at the same time as pushing to deliver the best approximation of the producer’s vision. It’s a checks and balance system in its truest form. Why expect these two halves to exist in a single person when you can instead hire two people each of who are uniquely suited for the positions. (Plus the arguments between the two heads are now externalized and are fun to watch.)

Quality Assurance

The last detail in initial team layout I’m going to discuss is quality assurance (QA) also known simply as test. Test is regularly the most overlooked aspect of game development. But as with everything I’ve mentioned so far as projects increase in size this is another detail that can no longer be ignored. With larger games come larger test requirements. The sheer number of options in online and especially massively multiplayer games creates test matrices that can barely be fathomed. This is why attaching testers to a team earlier is better and also why our industry as a whole must start treating QA as a viable job and long term career.

The first major step with integrating test is at a company level. Most companies sadly still view testers as 19 year olds that can barely spell. And unfortunately, if that’s who the company expects in the positions then that’s what the company will pay for and that’s what the company will hire. However, major software development companies like IBM and Microsoft have known for decades that software test is a career for mature QA experts. It’s a real job with a real skill set often staffed with people from engineering. A strong test group needs strong organizational management, a solid engineering department, and plenty of headcount. Normally, only the last item is supported by companies and only in the form of cheap contract labor. This reality needs to change across our entire industry.

If you can convince your company that you need to integrate test early, hire a few engineer testers. They are engineers that know how to build test cases in order to stress test bare code systems. They can architect automated testing suites to limit the human equation when dealing with massive test matrices. Sit them with a deliverable team and get them involved in the day to day business of the product. A test engineer working with his team should be involved in the architecting process with an eye towards automation and how certain decisions might increase or decrease required hands-on test time. A correct decision early in the engineering process can save hundreds of hours for testers at the end of the project.

If you found this post interesting you should checkout the full article in the book Massively Multi-Player Game Development 2.

Show Full Story

Describing Game Behavior with Use Cases

This is an excerpt from the Article

Describing Game Behavior with Use Cases
by Matthew Walker

Modern MMP development projects are very large, involving many participants with different skill sets. It is rare for the same person to handle both game design and programming tasks. A new challenge arises from this need for specialization: communication of game requirements from designer to programmer. Games involve complex interactions between entities, requiring sophisticated state management and event handling. The creative vision of the game design must be systematically mapped into a technical design model so that the final implementation represents the initial intent. Use cases provide an effective means for doing this.

What Are Use Cases?

Use cases define a model of system behavior that uses natural language within a simple structure that designers and programmers can use as their common frame of reference. One of the primary functions of use cases is to decompose a large, unwieldy system specification into manageable parts. These smaller parts are easily described and consequently, more easily implemented.

Each use case represents a discrete unit of behavior with a clearly defined scope, distinct steps, and unambiguous preconditions and postconditions. The clarity of each use case's scope aids in developing a modular, robust system design by defining where specific responsibilities lie. Using distinct steps to describe a behavior illustrates what should be done to achieve the goal of the use case and ultimately maps into program code. Stipulating preconditions and postconditions for each use case ensures that the state of the system is known both entering and exiting the use case, and exposes dependencies between use cases.

Why Use Them for MMP Development?

On the large teams typical of today’s MMP projects, designers, artists, and programmers must communicate regularly and clearly to coordinate on the product's creation. Much of this communication is between people of varying levels of technical expertise. The approach of using simple, but structured, natural language in use cases reduces ambiguity without making too many assumptions about each party's degree of understanding of technical details. Designers and programmers can collaborate to build a fairly robust picture of the game they are creating without becoming mired in jargon or implementation details. Later, programmers can take the picture formed by the use cases and transform it into code more easily and with a greater understanding of the designer's requirements than if they were trying to do so directly from a textual game design document.


How to Write a Use Case

Use cases have been used in software development outside the game industry since their introduction by Ivar Jacobson in the early 1990s. Since then, many authors have written about use cases, presenting many different approaches to writing them. These range from intensely formal, cross-referenced, document hierarchies to little more than short, but consistent, modular descriptions of functionality.

The goal of this article is to present an acceptably consistent and clear approach, one that has been used in real game development projects, to illustrate how thinking in terms of use cases can aid MMP game development. Our technique will be to use a simple and repeatable structure, concise terminology using everyday vocabulary, and unambiguous English sentences.

Identifying Use Cases

Behavior that constitutes a use case is sometimes hard to identify. What starts a use case, and when does it end, are questions we must explore at the outset. The identification of actors and events helps us do this.

Actors

Actors are entities that exhibit behavior and initiate interactions with, and within, the system being developed. Actors cause use cases to happen. Identifying actors helps us to identify where a use case begins. A simple list of actors with descriptions should suffice to encourage completeness and expose duplicates:

• Player Character (PC) - In-game representation of the player.
• Non-player Character (NPC) - Computer-controlled entity that shares many characteristics of the player character, including the ability to chat, trade, fight, and join a party with a PC.
• Monster - A computer-controlled entity that can only fight with a PC or NPC.

Often, actors are external to the system. Common examples include human users and other software programs. This point of view is useful when developing user interfaces and other non-immersive aspects of a game, and it is the most common point of view in traditional software engineering projects.

In game development, however, an extremely important concept is that of an actor being an entity within the game environment itself. Many games are open-ended simulations of some aspect of reality, and within such simulations, entities often interact in complex ways. In such immersive situations, it is more useful to model the player's character as an actor, rather than the player himself. At this level, it becomes clear that individual computer-controlled entities such as monsters, NPCs, and vehicles benefit from being treated as actors. When we use this approach, we are able to break complex chains of activity involving multiple entities into discrete interactions between pairs of actors.


Events

At the highest level, use cases derive from events initiated by actors. An effective way to start identifying use cases is by listing and describing these events. Simply listing all the events that can occur in a game is a great way to brainstorm our set of use cases.

We do this by identifying the actor that generated the event, and providing a one-sentence description of the event. There are a couple of reasons for using a single sentence for the description. First, one sentence is short enough to include in a list while remaining readable. This aids in remembering and referring to events by name when discussing development with team members. The second reason is to introduce a discipline that is easy to follow: an event is a single occurrence; one can easily describe a single event in one sentence. If it takes two or more sentences to describe what is happening, it is likely that we are describing more than one event. Combining multiple events in such a manner is likely to confuse one's understanding of what is really happening.

A sample list of events from our role-playing game, with descriptions:

• PC Attacks Monster - PC targets monster and engages in combat using the currently equipped weapon.
• PC Casts Spell on Monster- PC targets monster and casts a damaging spell on it.
• PC Trades with NPC - PC purchases an item from an NPC.
• PC Heals PC - PC casts a healing spell on another PC.
• Monster Attacks PC - Monster targets PC and engages in combat using the currently equipped weapon.
• PC Equips Weapon - PC retrieves a weapon from inventory and inserts it into the weapon slot.

The Basic Course

The core of the use case is its list of steps. These describe the details of the behavior we want to implement. We use them to show the order in which actions are to occur, any repetition, and the existence of conditional logic. The first step should be consistent with the start of the use case description, and the last should reflect the final concept mentioned in the description. This ensures that the set of steps remains within the scope established in the description statement. If we find that a step appears to fall outside the scope of the use case, we need to either remove the step, possibly to another use case, or redefine the scope of the use case.

An example set of steps that fall within the scope of our earlier description might be:

Basic Course:
1) Player character targets monster.
2) Player character attacks monster with current weapon.
3) Compute chance of successful attack.
4) Compute amount of damage.
5) Apply damage to monster.

Now, our use case is starting to take form. It is still very simple, having only five steps. This initial set of steps, according to Jacobson, is called the Basic Course. Yet, we have discovered additional details about our use case. In order to attack a monster, we actually have to identify it as a target first. And, to allow for more interesting gameplay, we probably want to introduce some probability of failure for the attack. Finally, the amount of damage dealt to the monster may be dependent on many variables, so we introduce a step to compute this value. Here, we are hinting at the existence of other behaviors that may have additional complexity to be explored.

If you found this post interesting you should checkout the full article in the book MMP Game Development.

Show Full Story

Asian Game Markets and Game Development

This is an excerpt from the Article

Asian Game Markets and Game Development – Mass Market for MMP Games
by Adi Gaash

The Main Players – Who Does What

First of all we have the game developers: small, medium and large-scale companies that design both graphics and story and that program the games as well. The games are usually based on an existing game engines and communication technology. The companies support the game product throughout its full life cycle: from the early stage of the game's development and up to its final implementation, support (patches, etc.) and development of future versions.

Most of the successful developers are local companies. This is not surprising, since they cater to the preferences and tastes of the local end users much better than foreign companies. The aesthetic differences are evident when you examine content and graphic design, for example Manga titles vs. Walt Disney titles or Shrek. Games in China are based on local mythology and historical events (such as The Three Kingdoms) which have great appeal to the local gamer. There are a few hundred local companies that develop exclusively for this MMP industry. Some of them try to export their games, but most of them are meant to be sold inside the Asia market.

The central place for game studios is Korea. Some of the Korean studios also have branches in other Asian countries, but generally most of the content comes from Korea. Some of the prominent companies in the local industry are NCsoft, Nexon and Webzen...


Next, we have the service providers. Unlike the standard developer-publisher food chain, here the Internet service provider takes the place of the traditional publisher. Business-wise this makes a lot of sense, as promotion for new content can be easily carried out by online platforms. For example, Chinese companies SINA and NETEASE provide e-mail and web hosting services, but by looking at their web sites, one can see that those companies' main business is massively multiplayer role-playing games (MMP RPGs).

The companies buying the local rights handle most of the service and support issues, and of course get most of the revenues. For funding, check NASDAQ for updates on some of the service providers. If this business is new to you, you will be surprised by the numbers and by the dramatic change this online Asian market has brought to the global market.

Last but not least, we have the Internet cafés. Unlike the common use of Internet café in the West, in the East an Internet café functions most of the time as a "gaming café," with most of the activities being focused on games. One can say that they have replaced the 80s' and the 90s' arcades with regard to computer gaming.

As this option for online gaming use became easier to implement, chains of Internet – gaming café quickly gained in popularity. You can find a franchise as easily as you can find a fast food joint. In some cases, Internet café chains function as service providers with regard to publishing new games; my previous development firm negotiated directly with them for the rights to the game we have been developing. One of the major Internet café chains may be equivalent to a major service provider in helping a game become popular and achieve market penetration.

Communication Infrastructure to Support Your Game!

The fast Internet infrastructure situation is promising, as most of the houses in the main cities are already connected to broadband Internet – and those that are not yet connected will most probably be connected in the near future. Funny as it may sound; a room in a cheap hotel in Seoul might not be equipped with towels and a bar of soap but will have a PC and a broadband connection. Taiwan for example published a plan last year to build the infrastructure to provide full broadband access to the whole island.

As you know there are other communication services that become increasingly popular, such as video on demand and videoconference calling. The demand for these services speeds up the building of the infrastructure. Thus infrastructure and telecommunication firms like China Telecom, China Networks and others promote this idea to provide their customers an attractively priced communication package consisting of telephone, TV and Internet. Service providers have already contacted telecommunications companies to include online gaming services in their content packages.

According to a DFC Intelligence report, "the rate of broadband penetration has grown even more rapidly in certain Asian markets, most notably China, Korea, Taiwan and Japan". Their relatively optimistic estimate of 30.6 million households in Asia with broadband connection by the end of 2003 was exceeded by almost 18%. Since they also state that "the countries with the highest broadband penetration are some of the hottest markets for online games", we can conclude that the potential for online games in the Asian market is very high.

If you found this post interesting you should checkout the full article in the book MMP Game Development.

Show Full Story

Principles of Casual MMO game Design

This is an excerpt from the Article

Toontown Online: Building Massively Multi-Player Games for the Masses
by Mike Goslin, Joe Shochet and Jesse Schell


Problem: Conflict is essential yet violence is forbidden.

Creating a MMP for the mass audience and for families in particular, has some special challenges. While it is nice to imagine a happy game world where all the players just walk around beautifying their environment and doing nice things for each other, the truth is that to create an interesting world, some kind of conflict is required. The challenge is to make the conflict meaningful and important to players, but not so intense that it turns away crucial segments of your audience, such as parents or girls. We approach this problem in the following ways:

Fight Enemies for the Right Reasons
Many games let players take treasures from defeated enemies. We generally avoided this in Toontown, because it is one thing to defeat an opponent to prevent him from taking over your town, and it is altogether a different thing to defeat him so that you can take his possessions. There were exceptions to this, such as cases where defeating a Cog lets you reclaim an item that was stolen from another Toon.

Teamwork
Letting the players team up in small groups against opponents feels more justified than fighting alone, because in addition to protecting yourself, you are fighting to protect the others on your team.


Problem: Too many choices can overwhelm the player.

Traditional MMP games let players create a character, and then just put them out in the world. It is then up to the player to discover what they need to do next. Casual gamers are often frustrated by this lack of structure. Since these same players are often unwilling to read a lengthy manual, the best approach is to build an instructional structure into the game itself.

Tutorial
A much better solution is to gradually lead the player into the depths of the game, starting out very simply, with very few choices, and gradually offering more choice to the player as they learn their way through the game. Toontown features an integrated Tutorial and Quest system that gradually transitions the player through the following steps:

1. Single player, non-interactive tutorial
2. Simple tasks that must be completed as part of the single-player tutorial
3. Introduction to multi-player features through interactive tutorial
4. Exposure to the multi-player world, while performing simple tasks
5. Simple multi-player tasks (making friends)
6. A series of gradually more complex game tasks selected by the player, and completed in the order that the player prefers

For most players, the tutorial will be their first taste of the game, and a designer only gets one chance to make a good first impression. Be sure to schedule enough time to design the tutorial wrong a few times, because you will. Weaving the tutorial into the game is a difficult design challenge, but it is well worth the extra effort of design, implementation, and play testing that it takes.

Quests
In Toontown we decided to give the player a series of explicitly stated quests throughout the course of the game so that there would not be any question about what the player was supposed to be doing. We even provide a Quest Page that clearly shows current quests, quest requirements, and current progress. To personalize the experience for each player, though, we implemented a tiered Quest Pool” system, so that each player gets choices between randomly selected quests. When the player has finished enough quests in one tier, they graduate to the next tier, which features more difficult quests. As the player advances, they are allowed to work on multiple quests simultaneously.

While long-term goals (Drive the Cogs out of Toontown) provide good context for the player, it is the series of short-term quests (Bring this bread to the baker”, Defeat three Pencil Pusher Cogs) that make the game addictive. However, climbing the same quest ladder all the time can grow tedious, so we have found it useful to have more than one way to progress in the game. In addition to the quest system, which gives out many of the progress rewards, such as improved laff meters and larger jelly bean jars, there are other ladders, such as a separate ladder for experience gained in battle. The battle ladder is completely separate from the quest system, and gives out a different set of rewards (new types of gags). There is also a fishing ladder, which helps players recover lost laff meter points. Although players will often find themselves climbing more than one ladder simultaneously, the ability to shift focus from one ladder of achievement to another helps keep the game interesting and give it variety and depth.


Problem: The audience is impatient for the fun to begin.

Hardcore gamers will put up with a lot in order to play a game. Casual gamers want to start enjoying their game right away. The quest system described above helps lead players toward the fun elements of the game, but Toontown also tries to inject fun into some of the more mandatory aspects of a MMP.

The Download
While Toontown is downloading, instead of asking the player to stare at a load bar, we created a simple Flash movie and game for players to enjoy while they were waiting for the initial download to finish. This was also a good chance to introduce some of the game elements and back-story.

Create-A-Toon
Designing your avatar is an amusing game of mix and match. Even picking a name (in some MMP games, a frustrating experience involving multiple rejected names) is fun due to the funny name generator” that suggests all kinds of crazy Toon names.

Teleporting
The act of teleporting is necessary to get around quickly. Toontown makes the act whimsical by using portable holes. Toons pull a portable hole out of their pockets, jump in, and pop out another hole at their destination.

If you found this post interesting you should checkout the full article in the book MMP Game Development.

Show Full Story