First of all, you should compile a list of the books that are valuable for you and you want to buy the first edition of these books.
Accept the fact that the rarer the book is, the harder it will be to find and buy the first edition of that book. So, be patient and never give up looking for it as you will get it by finding the right people.
Visit the thrift stores for searching the first edition of books that you want. In these stores, sometimes you find the books of your choice and these first edition books are usually cheaper as compared to the local merchant or collector.
While buying a book from a thrift store, make sure you do not show your interest in particular first edition. Deal with the owner of the book in a normal way otherwise he can raise the price by observing your anxiousness for that particular book.
You should have knowledge about book collectors of your area. Talk to them about those first edition books that you want. You might get valuable information from them about the certain locations or stores from where you can buy your required books.
Do not forget to visit local libraries in your area. Many libraries sell older books in order to accommodate new arrivals and it is quite possible that you find the first editions of your desired books in the selling lot of libraries.