If the entire amount of the Roth conversion is taxable income, then you could go ahead and just pay the additional amount due using one of the electronic payment methods at irs.gov. Make sure to select 1040 for 2023 as the form you're paying for. The IRS will figure out what happened when they do document matching and will probably send you a letter saying you owe $0. If you wait until after April 15th to pay, there will be an underpayment penalty. If you choose not to file an amended return, you do still need to file form 8606. That form can be filed alone, without a 1040. See the 8606 instructions for details.
If you don't want to do that, or if some part of your Roth conversion is not taxable, then amend your returns. If you did your returns yourself using software, then that same software should also be able to do an amendment and e-file it (state amendments may have to be filed on paper, it depends on the software and the state). Or you can use paper forms for both Fed and State and mail them in. The Federal amended return is a 1040X and you'll also need a new 1040 with the correct numbers on it. If you send in a paper return, you can still pay electronically, but make sure to select 1040X as the form you're paying for.