Not necessarily. Some common rocks can be scratched with your fingernail: shale, soapstone, gypsum rock, peat. Others may be soft in the ground, but they harden once they spend time in the air (and vice versa).
And there is an imperceptible gradation between consolidated rocks and unconsolidated sediments. Indeed, geologists name and map many formations that don't consist of rock at all. This is why geologists refer to work with igneous and metamorphic rocks as "hard-rock geology," opposed to "sedimentary petrology.