How can I automate a batch crop and straighten function on single pics?

Here's my problem. I have to shoot thousands of pics of baseball cards. The cards are of varying sizes but they're rectangular in shape. I need to crop out the background (black velvet) and maybe straighten the images (depending on how close I can get the images straight when I shoot them)

PS has a nice command that will automatically crop and straighten pics that have been scanned. I've tried to run this command on my single pics and it works fine, except that I also get two additional crops/pics of stray hairs on the black background. I've also tried to make an action that I can turn into a batch or droplet and run on all the pics. I've not had much success. I'll end up with pics of stray hairs and not the card. I then have to go and manually delete the stray pics. Too much work.

Is there any way I can automate the process of cropping, straightening the pics then saving then new version?

Thanks