The base of your Regency outfit is a shift or chemise, next comes the stays, then petticoat, chemisette (if daytime) and gown; and if outside, spencer or pelisse and bonnet.

Underneath everything comes a linen/cotton shift
Link for shift pattern

Then comes the stays (not a corset, this was for support and a little shaping, not like Victorian or modern corsets).
You have the choice of half stays or full stays.
Link for full stays pattern

Next comes a petticoat.
For an example pattern please see this link

Links for Regency gowns:
1804 Anne Taylor Day Dress Pattern
1810 Bodiced Petticoat Pattern
Regency Back Closing Gown
Regency Era Sewing Pattern
Regency Era Style Costume Dress & Jacket
Regency Round or Trained Gown with High Stomacher Front opening
Regency dress with front closure, apron, cap and fichu about 1810
Regency dress with sleeveless Spencer 1805 to 1810

Further websites for dress patterns
Daughters of American Revolution
Laughing Moon Mercantile
Black Snail Patterns
Sensibility Patterns

If outside in the summer months a spencer (or half jacket) would be advisable
Vena Cava Spencer pattern
Maccall Spencer pattern

Fabric Warehouses
Dalston Mills
AmoThreads
Kantha Decor – Block prints
In person for reenactment silks and linens ARM, see the websites for dates of market in March and November
In person for reenactment linens TORM, see the websites for dates of market in March and November
In person local to Oxfordshire – Masons in Abingdon