So I've had a closer look at this.
I have tried to simulate a weak power supply by limiting the voltage and current on my lab supply to values at which I expect stuff to barely work, namely 4.8V and 0.3A. With the latest build I get 0.19A current when the system is up, between 0.2A and 0.27A during boot-up, and a brief spike in which the current runs into the limiter.
When the system does not boot successfully, I get the typical issues that people have complained about, such as the garbled color screen (which is the firmware code rendered in 32-bit RGB, BTW), a white screen, a black screen or a hang at the BE logo.
There are two things that can be done to make the system work sometimes when limited to 0.3A and reliably when limited to 0.35A. The first is to disable the EHCI controllers. These are currently unused, and not bringing them out of reset saves just enough power for the system to survive the current spike and boot up successfully. I am not going to do that, however, because the EHCI controllers are required for USB2.0 support, and that is likely to happen at some point in time, and when it does I don't want to deal with complaints from people with marginal power supplies whose systems won't come up any longer.
The other possibility, and what I would suggest to people having issues, is to provide some support for the 3.3V line of the Orange Pi. Adding a substantial capacitor between pin 1 (3.3V) and pin 6 (GND) -- I have tried 1000 uF because that's the largest I happen to have lying around -- is good enough to make the system boot up reliably even with a supply that is way below specifications. It is worth noting that bypassing the 5V line does not do anything, which is why I believe this to be a design issue in the OPi power supply circuitry.
I would like to point out that I do not recommend running Engine BASIC on the H3 with a sub-par PSU. Any change in the firmware, such as exploiting additional CPU cores, new device drivers, or attaching more USB devices will have you run into trouble. If you have a supply that should be good enough on paper but doesn't work well in practice you might want to give this a shot, though.