Perhaps you are correct. In my brief search for some history, legends, and cultural perceptions of the whistle and related instruments in Ireland and Scotland, I came across this:
Principally seen as a rustic toy associated with shepherds and bucolic idylls, the whistle was generally not taken seriously in Scotland by dance masters, music teachers, publishers, etc. In 18th and 19th century tune collections, a great many of which advertised their suitability for the german flute, there are few references to whistle. Exceptions are The Caledonian Museum of c.1810, which contains tunes "adapted for the Flageolet" (a whistle-like instrument with various designs), and the 1800 Broderick & Wilkinson Selection, in which the tunes are adapted for the Harp, Pianoforte, Violin, or Tabor & Pipe (the tabor pipe is a three-hole whistle mainly used in English Morris music).
However, there is little doubt that the whistle was used by folk musicians. One such was Wee Willie White who busked the streets of Glasgow in the first half of the 19th century. A little later Carl Volti (born Archie Milligan in 1849), who became well known as a composer of classical music and fiddle tunes originally started with the whistle and formed a whistle band in his youth.
It does seem the mixture of simplicity, ease of playing, and perhaps also in recent times the use of the whistle as a child's or beginner's instrument has lead to a low reputation of the whistle as a "toy" rather than a serious instrument. For me the beauty of the sound of good whistle playing seems to make such a perception irrelevant.
On another note, we know that the low whistle is a modern invention, but I wonder if there were not older if not medieval or ancient precedents to the low whistle? Are there no legends or myths related to even the flute?