Who am I to write this one? But, really, it is what I observe in the last couple of years watching some open-sourced projects die slowly.
Start with something big. Imagine you want to create a next-gen music player, sort of Amarok-killer. Don’t worry about skinning, lastfm integration, support for Oracle DB, automation with infrared remote control etc. One step at a time. Just make your player plays MP3 files (or Ogg, pick anything you like) and continue from there. Keep the horizon in your vicinity so that the journey does not look like an impossible intergalactic travel.
Blind design. Don’t get me wrong, design phase is a very important one. However, if you are new to the field, don’t bother too much with the design because your project will need a rewrite anyway. If you never study physics/electrical engineering before and your boss asks you to build a transatlantic fiber optic transmission system, I bet you would need to redesign your system several times so at first no need to talk about transmission impairments, price, modularity, and other issues. If your first serious project is supposed to be a new revolutionary spreadsheet program, never mind about its scalability, multicore CPU support, and Excel interoperability, until you get the core spreadsheet functionalities right first. Design is important, and so is experience.
Smoke without fire. Unless your project is about a web application, then there is little point of discussing which CMS to use, where to host the website, how can we get the artists, etc etc. Sure, good looking web site, developers blog, and maintained forum can give more exposures to the project. But if you are only at the early stage, then just focus on the important part: the application. If you place your project on the host service like Google Code (avoid SourceForge), then the basic tools like subversion/wiki/issue tracker are already at your disposal. No need to show up your PHP skills there or to argue which artwork should be chosen for the logo, that can be done when finally your project has enough users. Spend your precious time doing something more helpful to the project’s important lifecycle.
Lack of passion. This is not work, this is supposed to be fun. Nobody holds a gun and tells you to do something. If you want to create a web browser but you hardly browse the web, forget it right away. Think twice: even if you are flamed and criticized, are you still going to love your new baby or not?
Too many mock-ups (will kill you). Yes, mock-ups are wonderful, they help people to visualize the end result of the project. But mock-up only without any code will not drive the project to success. You can’t post hundreds of mock-ups and hope that one or two real coders would step in and implement everything for you. In the open-source world, it is close to impossible to ask people to do what you want (unless you have billions to spare).
Follow the fashion. If you want the fame, do not waste your time doing software and do something more sensible (e.g. practice a lot for the next year’s Idol program). The problem with cool stuff is, when it is not the trend anymore, you are deserted. So stick with common sense and do what you think you can do best, not always what others think must be urgently done. Imagine if we have a dozen of PointCast clones…
Reinvent the wheels. Yes, this is so obvious. But let us talk about the components here. For example, even if you hate STL, just use it while you are kickstarting your project. If your design is correct, you can replace it later on (often not trivial, but still better than wasting time creating another library). When you need scripting support, just pick Lua/JavaScript/Python/etc at first. If finally it is absolutely necessary, you can hook your own script interpreter. Focus on building the application, not all the nitpicky details which can be improved with time.
Underestimate the maintenance. Just check SF, how many projects did become orphaned in the last years? You can create two dozens open-source projects if you want, but if you think you won’t be able to maintain them, then think about it again. Most of the time, we always hope that someone will take over the maintenance, but only a handful projects manage this. I assume, partly because maintaining something is not the fun part of the game. You may get slashdotted when magically you create something out of nothing, while when you fix a bug it is often (sometimes not at all) noticed only by the bug reporter.
Wanna add some other points?
(Don’t be trapped in the fallacy: if you see these signs on the project you are working on, it does not mean that your project is bound to fail).