She didn't choose to violate genre conventions. She doesn't read fantasy in the first place, and Harry Potter wasn't intended as a (high) fantasy.

In an interview with The New York Times, J. K. Rowling said the following:

Any literary genre you simply can’t be bothered with?

“Can’t be bothered with” isn’t a phrase I’d use, because my reading tastes are pretty catholic [lowercase in original]. I don’t read “chick lit,” fantasy or science fiction but I’ll give any book a chance if it’s lying there and I’ve got half an hour to kill.[...]

She also didn't consider Harry Potter and the Philosopher's Stone a work of fantasy until after she'd written it. Another answer to the linked question contains numerous additional quotations indicating that Rowling doesn't like or read fantasy, so I'm going to consider that a well-established fact.

Given that Rowling does not read fantasy, we have to interpret Harry Potter in the broader context of English literature rather than as an intentional work of high fantasy. Comparisons with The Hobbit and other works in Tolkien's legendarium are particularly inapt, because those works spawned a literary and cultural canon which Rowling has intentionally avoided.

Dragons are, of course, fearsome creatures. But there is also a longstanding tradition, far older than Tolkien, of individual knights slaying them. Probably the earliest surviving example which can be called "English" in any sense is Beowulf, but of course there are numerous more modern examples. The dragons used in the Triwizard Tournament are a continuation of that literary trope. The "knights" are schoolchildren because the protagonists are schoolchildren. Finally, they didn't actually slay the dragons. They just had to retrieve an egg.

Turning to broader genre differences, it's clear that Rowling was intent on writing a "civilized" world, in which humans have a substantial level of control over magic. The dragons (and various other magical creatures) are kept as pets. The elves are enslaved. The goblins once rebelled, but now they merely run the economy. Why? Because Rowling was not writing a fantasy in which "anything can happen." She was writing a boarding school story which just so happened to be set in a magical environment. Boarding school stories can have a substantial level of mischief (think of Harry and company sneaking around with the invisibility cloak), but a common baseline assumption is that the school is mostly safe and mostly a civilized environment, in which children need only moderate adult supervision.

Dragons in Harry Potter are indeed powerful.

Though it’s true that dragons were used in one of the tasks of the Triwizard Tournament competed in by schoolchildren, dragons were shown to be powerful creatures. The task wasn’t actually to defeat a dragon, just to retrieve an egg guarded by one. To defeat a dragon, Sirius Black says would need about six wizards working together.

“There’s a way, Harry. Don’t be tempted to try a Stunning Spell – dragons are strong and too powerfully magical to be knocked out by a single Stunner. You need about half-a-dozen wizards at a time to overcome a dragon –”
_{^{- Harry Potter and the Goblet of Fire, Chapter 19 (The Hungarian Horntail)}}

Dragons are also classified as XXXXX, the highest of the Ministry’s danger classifications, which means known wizard killer/impossible to train or domesticate.

“DRAGON

M.O.M. Classification: XXXXX

Probably the most famous of all magical beasts, dragons are among the most difficult to hide. The female is generally larger and more aggressive than the male, though neither should be approached by any but highly skilled and trained wizards.”
_{^{- Fantastic Beasts and Where to Find Them}}

Though dragons may be portrayed as stronger in other works, they’re not portrayed as weak in the Harry Potter series. As for why they’re not portrayed as even more powerful, that may be because dragons aren’t as large a part of the story as they are in other works.

They can decimate entire armies without taking much damages

They could decimate entire medieval armies without taking much damage. They would be less successful against a fighter plane or two. There is nothing about the traditional fantasy dragon that would give it much of a chance against a modern military.

Now if muggle authorities can handle dragons, it would be something of an embarrassment to the wizarding world if they could not also deal with dragons. That would make the whole wizarding thing much less awesome. It's true that muggles do a lot of things better than wizards, but dragon slaying is too much.

There is also the general principle that if you don't want an unstoppable monster to be the focus of your story, you don't have one period.

Harry Potter takes place in the modern era, rather than medieval times, where the natural world has over centuries become "civilized".

Dragons are indeed powerful, and it's because of this power they are downplayed in the Harry Potter stories. In fact, dragons were so powerful they disrupted the spread of the noble wizard and muggle, and as a result were slowly hunted and pushed to the brink of extinction, much like North American Bison. By the time we meet Harry Potter, they're just not something one is ever likely to encounter in the normal course of things. If not for the work of Dumbledore and Nicholas Flamel finding ways to make Dragons at least somewhat useful, they might be gone completely.

Dragons in fantasy works ... can decimate entire armies without taking much damage.

True, true. But so could wizards. For the tournament, the dragon is chained and at least partially trained, and the goal is merely to steal from it rather than defeat it. And these aren't just children. The tournament is only open those at least 17 years of age... practically adults.

It should also be considered that in the Harry Potter universe, what Muggles "know" about the magical world is often distorted or downright false.

HP gnomes look nothing like the gnomes in that coffee-table book from a few decades back, elves look nothing like Galadriel, and so forth.

In the case of dragons, we have intelligent and articulate dragons in Muggle fiction (such as Smaug in The Hobbit), but in the HP universe dragons' intellect is on the same order as that of lower animals. So more has been changed other than their raw power and ferocity.