The phone's manufacturer (Google in my case) may stop pushing firmware updates, allowing vulnerabilities to remain unfixed and exploitable.
The custom ROM's maintainers may lose interest in the project, or the project may stop supporting the phone in question, again allowing vulnerabilities (this time in Android's source code) to remain open.
If by "people sorting" you mean "facial recognition" - well, it should "just work". You may, in the admin backend, go to the jobs section and manually force it to start another scan for missing faces.
That said, it will currently only do facial recognition on the faces in the photos your account owns. If you're using partner sharing (e.g. with your spouse), then you will have separate facial recognition data (only done on your photos) and your spouse will have their own facial recognition data (done on the photos owned by them). Bottom line: facial recognition data is not "shared". If your spouse "owns" all the family photos in your immich, this is why you only see the coworker meme faces and not your family.
The same is true for memories.
The situation is unsatisfactory at the moment, but I've talked to the devs and it's on their roadmap, so this will be addressed in an upcoming release.