Semantic search with filters

Design search for a real-estate listings site where users issue natural-language queries ('quiet 3-bed near good schools under 600k') that must combine semantic similarity with hard structured filters (price ≤ 600k, beds = 3, status = active) and ranking by recency and proximity. There are ~5M active listings, constantly changing (sold, price-dropped, new). Design a hybrid retrieval system that respects the hard filters exactly while still using semantic matching, and explain why pure vector search alone fails here.