There are several ways. One way in rules is to use the trigger “SharePoint group membership”. You then use the group number which is found in the address bar when viewing the group. I would probably use “if not in a group” and then in the hidden tabs section use the tab unique ID.
Another way is in the tab itself. There’s a section called “Which groups can see this tab?”. There you can specify which groups have access.