To be safe, all harnesses, whether integral with a PFD or not, should fit properly.
This may be an issue if you are smaller than average, as noted below.
Regarding the fit of a sailing harness, the core instructions are: the attachment point of a harness must be “above the lowest point of the rib cage” (from ISO 12401). US Sailing has expanded on the ISO rule to note that many inflatable PFDs with a built-in harness are designed for people 5’6” in height or greater. If you are below that height there may be a risk of broken ribs or back injury if sudden tension is applied to the harness. It is strongly recommended that each person who is below the 5’6” height test their harness to insure that it is high enough. If not, that person should switch to a PFD with built-in harness unit that has a better fit, adjustable features, or an independent adjustable harness.