Two kinds of problem are distinguished: the first of finding processes which produce complex outcomes from the interaction of simple parts, and the second of finding which process resulted in an observed complex outcome.  The former I call the easy complexity problem and the later the hard complexity problem.  It is often assumed that progress with the easy problem will aid process with the hard problem.  However this assumes that the “reverse engineering” problem, of determining the process from the outcomes is feasible.  Taking a couple of simple models of reverse engineering, I show that this task is infeasible in the general case.  Hence it cannot be assumed that reverse engineering is possible, and hence that most of the time progress on the easy problem will not help with the hard problem unless there are special properties of a particular set of processes that make it feasible.  Assuming that complexity science is not merely an academic “game” and given the analysis of this paper, some criteria for the kinds of paper that have a reasonable chance of being eventually useful for understanding observed complex systems are outlined.  Many complexity papers do not fare well against these critieria.