Issue

You can actually install a trustworthy boot fimrware on your computer yourself and actually verify that it really is.
The issue is that you would have to do it each time you use a computer this is very unpractical. Since you would need an additional computer to copy it from, it would shift the trust that additional computer, which would have the same issue.

Why this is problematic?

Can you trust your computer if its software has been arbitrarly modified? As the software that runs first, and that is the most privileged, the boot fimrware could modify the software that runs after or along it.

Bootroms

Lot of devices using system on a chip can be configured to refuse to unsigned boot firmwares. This is however unreversible. The users opting for such scheme would have to either delegate that trust forever or to maintain their own PKI. If they chose the later, loosing the signing key would prevent the installation of new boot fimrwares. If the signing key is exposed, it would void the boot fimrware security.

Chromebooks

With chromebooks, the first stage of the boot fimrware is typically made read-only. It then checks the signature of the next stage, and enters recovery if it doesn't match.

Lot of effort is made to ensure that the users are not prevented to load the operating system that they whish. With some devices, there is even some facility to permit the user to easily run a coreboot payload such as SeaBIOS. However totally changing the boot fimrware requires the user to disassemble the device to remove the flash write protection. Even if this is made easy, it is unpractical for a lot of users.

TPM:

A TPM is like a smartcard, with some additional features, such as the ability to "measure" data.

That measurement is typically used by boot fimrwares: The stage of the boot firmware that is running sends the next stage to the TPM (and optionally some configuration data) to be "measured".

The TPM then hashes that data (with SHA1) and appends it to the previous hash.

It can then be configured to release a secret only if the hash matches a given value.

If TPM are not built into the processor, they can be circumvented by making the TPM think that the computer is booting, or by running a modified boot firmware, and in both case feeding it the original firmware pieces and configuration to retrieve the secret. While this attack might be difficult to carry out in practice, it would be totally invisible to users when successful.

With recent Intel processors, TPM are inside the processor. However, they have become an application running on the management engine firmware. This firmware is proprietary software and signed. And you are not even supposed to have access to it.

New approach?

Another approach would be a (separate) chip that would verify the boot firmware signature and only release a secret if it matches. The user would be able to change the public key at will. This would also erase the previous secret, and the user would have to re-add one.

Such mecanism could be used to display a secret picture to the user, attesting that the system is in a trustworthy state. This picture wound't be available if the system is untrustworthy. That way the user would for instance only their passphrase when the picture matches.

It could also release an encryption key that would unlock some data partition. The key would be lost if the boot fimrware is modified.

This way the users would be able to delegete the trust to whoever they whish, and still be able to reverse their decision when they whish.