If there are no creatures in the player’s library, then the target creature is still destroyed, you see all the cards in that player’s library, and then they shuffle and continue play.

A “creature card” is any card with the type creature, even if it has other types such as artifact, enchantment, or land. Older cards of type summon are also creature cards.

If the targeted creature is an illegal target by the time Polymorph would resolve, the entire spell doesn’t resolve. Nothing else happens.

If there are no creature cards in the player’s library, all the cards in that library are revealed, then the library is shuffled. (The targeted creature remains destroyed.)

If the targeted creature has indestructible, it’s still a legal target — it just isn’t destroyed. The rest of Polymorph’s effect happens as normal.