Reader mode loads the page from a local copy cached from the non-reader mode connection; via localhost, which doesn't support https. (or rather, uses a self-signed cert which appears insecure). As this connection is entirely internal to the device, proper https isn't necessary.
Later versions of the app will hide that icon in reader mode, but it doesn't look like that's made it to android yet.
it might be related to the root certificates available on your device. you might be able to click the barred icon and read details about the error that's encountered