Lemmy is a federated network (part of the Fediverse), meaning that, for the most part, all instances are linked together and share most of the same information and comments. If people move from one instance to another then their experience will be almost identical.
If you already have an account on Lemmy.world then you can transfer your account information to an account on a different instance by using one of the many tools created by Lemmy's users. The reason you would move away from Lemmy.world is that it helps decentralize Lemmy's data, so that if one instance goes down it doesn't have a large impact on others' Lemmy experience.
If you are using an instance with only 10 users, then here are some issues you might run into: there are less users to update the instance's 'communities' page, so you might have to add some communities to the instance manually; there aren't as many people maintaining the instance, so it might not be as well-maintained as some larger instances; the server costs might not be worth hosting for just 10 users (donate to your host, if you have the means to do so). However, there are a lot of smaller instances that are just as well-maintained as Lemmy.world (like Midwest.social).
I hope this helped you understand Lemmy a little better.