Yes, there is a substantial difference in weight (mass) between male and female Ospreys, as for most raptors. Females are uniformly larger than males. In fact, this is how sex of nestlings is usually determined, although definitive sex determination can be best determined by DNA techniques to detect differences in the sex chromosomes.
It should be noted that unlike in mammals, female birds are heterogametic in having Z and W chromosomes. Males are homogametic with two Z chromosomes.
Due to the rapid growth, hatch order is initially reflected in the size of the nestlings during the early nestling period. Eventually, females may surpass earlier-hatched males in size, or the disparity in size may become greater for earlier-hatched females. Once growing juvenile feathers become prominent, they may confound visual estimation of size. After nestlings have achieved maximal body size (about the same as adults), there is still much feather growth to occur. In fact, flight feather length (retrices/tail, remiges/wing) is incomplete even upon fledging. Also, feather growth rate does not differ between males and females.
Here is a reference paper by Schaadt and Bird describing the results of numerous morphometric measurements of a sizable number of nestlings throughout their development. The same measurements were made on adults. The main data can be found in Figure 1 and Table 4. Some particularly interesting sections have been highlighted.
The data Schaadt and Bird provided—which shows measurements of a substantial number of nestlings throughout the nesting period—was re-plotted, with points added for the weight (mass) of all the nestlings that have been banded at this nest. Values for this nest were then plotted along with age at banding. The values correspond pretty well to either the female or male curves, with only one somewhat equivocal case (VW from 2018). This chart also shows that hatch order does not correspond to size after a few weeks. This nest appears to have little or no food limitation.