Backreaction has provoked a discussion about the role of thought experiments in physics.
Well, they have been really important for quite some time!
The gedanken experiments, as they're often called, have been produced en masse since the first examples given by Hans Christian Ørsted who also gave them the name back in the early 1820s. They're situations that we imagine - but don't immediately realize - and that we analyze as carefully as we would analyze a real physical situation, in order to find a paradox, resolve a paradox, find a limitation of the existing approximate laws of physics, or find a universal physical principle.
Don't get me wrong. In principle, physics could make a lot of progress even without this theoretical method.
Except that in reality, the physical phenomena are organized at so many vastly distinct scales that we constantly need to imagine that various quantities are jumping over many orders of magnitude and we're getting to extreme regimes where the extrapolations of the well-known approximate laws and our experience can clash with each other. That's often possible only in theory.
Starting from mechanics
Occasionally, gedanken experiments become actual, real physical experiments when the technology gets much better: the Feynman inverse sprinkler is a well-known example of a thought experiment that can rather easily become a real one (yes, this one, in the air, did rotate in the opposite direction).
Newton's Cannonball is another. However, I think it's equally important to notice that even in the situations when the technology doesn't get that far, we may fully crack a theory that determines what would actually happen if the gedanken experiment were realized.
When people were thinking about thermodynamics, some of them wanted to produce perpetuum mobile devices of the first kind or the second kind. The former violated the energy conservation law while the second violated the second law of thermodynamics, one about the increasing entropy.
In the second category, we find many versions of Maxwell's Daemon, for example the Brownian Ratchet due to Feynman. They're hypothetical devices that have been believed by some people to be able to produce unlimited energy.
However, when you analyze their function properly and in detail, you will find out that they don't work. A crucial aspect of the situation is that you don't actually have to construct the device. It is enough to analyze it purely theoretically and decompose its functions into individual pieces (and their interplays). Once you do so, you will find out that these machines can't work.
Well, if you believe that it's impossible to prove anything about a machine you haven't actually constructed, I can weaken the statement for you. By a detailed theoretical analysis of the gadgets, you may show why another theoretical argument that suggested that these machines should work has been incorrect from the beginning. To say the least, this reduces the probability that such a machine actually can work. ;-)
As you can see, gedanken experiments force us to think more carefully than we would think if we were just satisfied with a vague proposal for a future gadget or with a "null hypothesis" about the physical laws that govern so far untested situations. They often tell us that certain laws - or intuition - can't be extrapolated too far. And sometimes they can.
Modern physics: relativity, quantum mechanics
Einstein's contributions to physics were almost completely theoretical. And because they were revolutionary at the same moment, it's clear that he couldn't have achieved those things without thought experiments.
When he was a teenager, around 16 years of age, he may have been imagining nude women every 50 seconds, or whatever the statistics says. But more importantly, he was also imagining observers who are are trying to catch up with light. Clearly, no available vehicle was able to move by the speed of light, or anything close to it, but Einstein was simply captivated by the question what happens.
Newtonian mechanics would allow the observer to match the speed of light. However, Maxwell's equations indicated that light should be moving by the speed of light - and it looked like those equations should hold even in the observer's reference frame, so the speed should be "c" with respect to anyone and anything. In 1905, Einstein finally solved these matters in the patent office.
He was imagining various synchronization problems with trains that are close to the speed of light, and so on. In the context of general relativity, there's a lot of crucial thought experiments. The Einstein lift sheds light on the equivalence principle. Various configurations of masses that collapse into black holes are modern examples in the context of general relativity.
Quantum mechanics: just a list
Quantum mechanics brings its own collection of gedanken experiments - such as the EPR paradox, Schrödinger's Cat, double-slit experiment, Elitzur-Vaidman bomb-tester, GHZ experiment, Heisenberg's microscope, Popper's experiment, quantum pseudo-telepathy (violations of Bell's inequalities), quantum suicide (in the many worlds picture), Mott problem and Renninger negative-result experiment, Wheeler's delayed choice experiment, Wigner's friend, and others.
In the context of quantum gravity, most well-known experiments deal with the information loss paradox, in one way or another. People are imagining a lot of observers doing ordinary as well as crazy things while falling into black holes, and so on.
Again, it's important to note that some of those experiments have actually been realized while others have not. In the cases that have already been observed, the correct answer is typically viewed as a fact that everyone is forced to accept, even if her expectations were different.
However, that doesn't mean that the results of the gedanken experiments that have not been made yet are unknown. They're often equally - or even more reliably - known than the results of the realized gedanken experiments. The correct answers follow from the same well-known theories.
And of course, some people who don't quite understand these theories often end up with wrong answers or at least wrong interpretations of these gedanken experiments and "facts" can't immediately convince them that they're doing something wrong because the "directly known facts" are very indirectly connected with the theoretical questions we want to ask. They are connected by chains of logical and mathematical reasoning and only good enough theorists can safely walk on these chains.
In fact, most of the thought experiments I listed above were proposed and are still being abused as memes to promote misconceptions about physics most of the time. The known answers to the question "what actually happens" is often very different from the "popular" answer that is being associated with the meme by most pundits.
I have mentioned that people have prepared many gedanken experiments concerned with the black holes, even in the quantum regime where the information loss paradox has to be addressed. Some of these experiments have been fully understood - for example, we know for sure that the Hawking radiation of AdS black holes subtly depends on the initial state - while others have not - we don't have e.g. a "quasi-local" algorithm how to decode this radiation and it's conceivable that no acceptable method exists and no "sharp paradox" can ever be localized.
A lack of thought experiments
However, I kind of feel that the black hole information paradox is not the only class of conceptually difficult situations that must be fully understood for us to crack the theory of everything in its entirety. In other words, I feel that we haven't seen a sufficient number of thought experiments in high-energy physics.
This is not a problem that began in string theory. It began in normal quantum field theory. And this problem is actually the "other face" of the extreme reliability and completeness of the framework(s). Quantum field theory is a framework that produces "theories of almost everything", using Lisa Randall's terminology. It shouldn't be shocking that a "theory of almost everything" answers many more questions than it opens.
And indeed, that's the case of QFT and perturbative string theory, too. They're just telling us what kinds of questions are legitimate and how to answer them. Usually, we are led to believe that all meaningful physics is encoded in scattering amplitudes. And QFT as well as string theory are giving us recipes how to calculate them.
Scattering amplitudes follow a very universal logic and seem to be straightforward and paradox-free in consistent theories. Nevertheless, they're probably not the only types of questions we should really master before we deserve a PhD in the theory of everything. ;-) While the full information about the dynamics may be encoded in the scattering amplitudes or correlators, the exact ways how this information is linked to answers to various non-scattering questions and generalized scattering questions can still bring us some surprises.
While we know that the space (and time?) are kind of emergent in string theory i.e. quantum gravity, we don't really have too many good yet unanswered thought experiments that would attack our ignorance about some remaining conceptual puzzles of string theory. People haven't considered too many sufficiently difficult yet sufficiently well-defined situations in excited states of well-established stringy vacua whose resolution would tell us something we don't know about the birth, death, and radical transformations of space and time.
In this sense, I would argue that physics has a lot of solid answers to many questions - questions that seem unanswerable to a huge majority of the laymen (we have already made quite a few steps beyond the "theory of almost everything") - it is lacking some good new questions. It may be more difficult to find the right questions than the answers to these questions. And once we do so, we may do the next big - and perhaps final - steps to our complete mastery of the theory of everything.
And that's the memo.