Source: https://www.opensc-project.org/opensc/wiki/OpenPGP

Gnome's stock GPG agent is a piece of crap that doesn't support smartcards. This is an old known problem (2008 AFAIK), but gnome dev team did not give a heck to this. The most appropriate solution is to disable gnome's GPG agent and, if necessary, to properly configure your own gpg agent. To do so, run the following program:

gnome-session-properties

uncheck Gnome keyring Agent, then reboot or logout and login back.

apt-get install pcscd gnupg2 gpgsm scdaemon

Touch ~/.gnupg/gpg-agent.conf, then put this into ~/.gnupg/scdaemon.conf

reader-port "Gemalto USB Shell Token V2 01 00"
disable-ccid

gpg selects the first card reader it sees by defaut, if the first reader doesn't have a card, it will fail as follow:

13:37 user@hostname ~ % gpg --card-status      
gpg: detected reader `Dell Dell Smart Card Reader Keyboard 00 00'
gpg: detected reader `Gemalto USB Shell Token V2 01 00'
gpg: apdu_send_simple(0) failed: no card

Append –reader-port “reader name” to gpg command for version.