I was explaining to a professor the phenomena that I, a graduate student, was studying, describing to him the peculiar features of the data. “So what causes this?” he asked. “No one knows,” I replied. The professor got visibly angry and said ”there are always theories!“ Of course, there were theories for the phenomena , because physicists abhor a vacuum, and so they rush to fill the empty spaces of our knowledge with the air of mathematics. But the truth was that no one knew the cause of the phenomena, and in the end all of the proposed theories proved wrong.
When confronted with an unknown phenomena, theorists rush out numerous explanations. Most theorists rush out several very simple and obvious ideas within days of a phenomena's discovery, providing the canonical theories for that phenomena; most efforts within the scientific community are directed towards these theories. Over a longer period of time a handful of theorist develop less obvious, better developed theories that have a narrower appeal within the community; generally these theories are pursued by only several researchers, or perhaps by only the originator of the theory. Finally, there is a continuous flow of extremely speculative ideas, generally spun by the most mathematically-inclined theorists and often involving imaginary objects, that provide some entertainment to the community, but no worthwhile results. What we never have is the theoretical community stepping back and pausing until better data arrives.
Two problems almost always confront theorists when a new phenomena is seen: sparse data and complex physics. The data is usually sparse, because we only can observe the light from an object, and often this light is confined to a single frequency band—we see x-rays from an object, but we don't immediately see optical light. The physics is usually complex because in general the phenomena producing the light involves physics that is difficult to simulate on a computer and too exotic to replicate here on Earth.
The first time an object is seen, it is often seen at only one frequency. For instance, the first observation of an x-ray source by a satellite might not be immediately accompanied by an optical observation of the source. Normally such an observation is insufficient to guide the theorist in creating a theory. Normally we need an optical counterpart at the position of the x-ray source. Without an object at optical frequencies, we are usually at a loss even to know whether the object is within the Galaxy or much farther away in another galaxy.
The light we see from astronomical objects is often produced through a complex and exoti set of physical processes. Some of the physics is as complex as the weather here on Earth. This type of physics generally requires a computer to solve. For instance, the conditions in many astronomical sources is right for the convection of very high-temperature gases. These gases are turbulent and ionized, which means that they generate magnetic fields. These magnetic fields in turn affect the viscosity of the fluid, the heating of the fluid, and the generation of electromagnetic radiation by the fluid. We see this at the surface of the Sun. Around other types of stars, this physics can be more exotic. The most extreme example of this is the physics of fluids within a neutron star, which is the remnant of a supernova explosion. At the surface of a neutron star, the force of gravity is so strong that it must be described using general relativity. The magnetic fields at a neutron star's surface can be hundreds of thousands of times stronger than the strongest magnetic field produced in the laboratory. The physics in these examples cannot really be solved by computer without strong experimental and observational guidance, but this guidance is often absent.
A simple example of these problems is the structure of any astronomical object. Most objects cannot be resolved, appearing only as a point on the sky. Imagine if we saw the Sun like we see all other stars. Would we place Sun spots onto its surface? Could we understand the complex, twisting magnetic fields at the Sun's photosphere? While we have the Sun to give us clues to the appearance of other stars, many astronomical objects appear only as points of light, so we have no direct observation of their appearance. The geometry of such systems must be derived through other means, or just simply guessed at.
Years ago we were in this difficulty with the gamma-ray bursts. We had dozens of observations of sudden spikes of gamma-rays emission from random positions on the sky, but all efforts failed to find persistent optical or x-ray counterparts. What existed were time-dependent measurements of the rate at which gamma-rays were emitted and rather poor gamma-ray spectra. With so little information, theorists were only lightly constrained; theories placed the gamma-ray sources in all corners of the universe, ranging from exotic objects within our own Solar System to objects within the most distant galaxies. Once a counterpart was found in the late 1990s, we learned that supernovae in the most distant galaxies were the sources of gamma-ray bursts.
Inevitably the need for guidance makes theory a laggard behind the observations. At minimum, a theorist cannot create a theory without understanding the distance to the object, because without a distance, we have no knowledge of how much power must be generated. For instance, an object in one of the most distant galaxies will be radiating a trillion times the energy as an equally-bright (the same apparent brightness on the sky) object at the center of our Galaxy. This is the difference between the supernova of a whole star to the nuclear fusion in a thin layer of a neutron star. Once a distance is established, so that an energy source can be established, more observations are needed to pin down other pieces of physics. What is the geometry of the source? How is the radiation created? What causes the source to be variable? How is energy released within the source? This physics can only be established through observations, because the physics is too difficult to untangle through pure reason and computer simulation. For this reasons, the early observations of any phenomena are of either limited or no value in developing a theory for the phenomena.
The creation of the first theories for a phenomena more resembles the handicapping of a horse in a race than the disciplined exercise of pure reason. The first theories are in fact markers. Most theorists place their wagers of time and prestige on the favorites, preferring the safety of numbers to cushion a loss. The speculators among theorists place their wagers on the long shots in the faint hopes of contrarian glory. But how ever the outcome, you can bet that once new data has settled the race, the theorists will immediately stake their claims on the next new phenomena uncovered by the observers.