First of all, look for the receipt that the IRS provided when you applied for the EIN. The receipt will surely have the number.
Without an EIN, nobody can apply for the business checking account. So, if you have the account, the number would have been used at the time of applying. Call your bank to access the EIN.
Just like the business checking account, the state and local business licenses need an EIN. If you have any documents relating to the licenses, you can easily get the EIN back.
Checking your previous tax returns is probably the easiest way to find the EIN.
If you have no previous record, you are left with no option, but to call the IRS Business & Speciality Tax Line. The department may give you the number over phone, if you provide the required information.