That's explicitly spelled out in the documentation for count (emphasis mine)
An implementation may choose to not execute the stream pipeline (either sequentially or in parallel) if it is capable of computing the count directly from the stream source. In such cases no source elements will be traversed and no intermediate operations will be evaluated. Behavioral parameters with side-effects, which are strongly discouraged except for harmless cases such as debugging, may be affected. For example, consider the following stream:
List<String> l = Arrays.asList("A", "B", "C", "D");long count = l.stream().peek(System.out::println).count();
The number of elements covered by the stream source, a List, is known and the intermediate operation, peek, does not inject into or remove elements from the stream (as may be the case for flatMap or filter operations). Thus the count is the size of the List and there is no need to execute the pipeline and, as a side-effect, print out the list elements.
It looks like filter interferes enough with this ability to require the execution of the side effects inside map.