The Linux directory system is a single tree from the root /. You can mount any filesystem to any directory inside it to extend it and have all writes to that location be handled by that FS. This is all irrespective of what filesystem the is present at that location in the tree. It does not matter if it is BTRFS, ext4 or anything else mounting a filesystem into the directory structure is handled by the kernel separately from the FS implementation. So, yes, you can mount any partition that contains a filesystem to /home/user no matter what you have done with / or even /home.
But, any writes to that location will be handled by the filesystem driver for that partition. So any subvolumes or anything else the main filesystem/partition has wont be available inside that directory. You can have a BTRFS filesystem mounted there from a separate partition if you want. Though a big benefit of BTRFS is the ability to use subvolumes instead of full partitions so you are not segregating the space on the disk (ie, any subvolume can use what space it requires and you wont have one running out of space because you didn't make it large enough). So if you are going for BTRFS subvolumes I would just have one main partition and use subvolumes to split up the space if you wish. Though really the only benefit to that is you can snapshot them separately and I think you can set different quotas and settings on each one.
Thanks for the detailed answer. If i go with one partition/multiple subvolumes, is it possible create a subvolume for my "user" home that is encrypted with luks?
subvolumes are integral to btrfs. You cannot have a layer like luks between them. You can encrypt the whole partition with luks before btrfs or you can encrypt specific directories after btrfs with something like encfs or truecrypt, though doing so loses some of the benefits of btrfs as it can no longer see your individual files.
If you wanted just /home encrypted with luks it would need to be a separate partition (which you could then have btrfs inside with subvolumns on that). Though IMO that gets a bit complicated - I would just opt for encrypting everything (except boot) on the root partition and have one btrfs fs on that partition with as many subvolumes inside that as you like.