First of all, love the idea of the category feature!
To my point: I noticed that making a category relevant to the All learners group overrules individual course visibility settings. My request is to make the individual course visibility settings overrule the category being added as relevant content to a group.
My case is that I want to have a category, let's call it "Agile", to make it easy for our users to find all content on the topic of agile. The category will contain a mix of internal content and marketplace content. I also want to separate our internal employees from our contractors, as the marketplace content is only available for our own employees. To handle this, I have two smart groups based on employment type. All the marketplace courses are set to "available in library" only for the group of all employees, which usually works well to hide them from contractors in the learning library. However, when adding the courses to a category and then adding the category as relevant content for the All learners group, the individual course settings are overruled - contractors can now see all the content within the category, including the marketplace content.
This is not specific for marketplace content and/or employees/contractors - the situation would be the same if you'd want to put only bridge courses on the same topic in one category, but have some of the courses on the topic only visible e.g. for managers.
Current workaround is to create separate categories ("Agile 2", "Agile for contractors" etc) for content that is really the same category.