I can appreciate your question. Exotic cars spend thousands of dollars to reduce weight by a few pounds. I suppose the next question would be how much a reduction of 40lbs would improve performance. If it were a 1/10nth of second in a 0-60 time, would that be worth the effort? Could you feel that on the road? For reference, the difference between a full tank of fuel and an empty tank of fuel is about 100 pounds or twice the weight of the A/C system. Does your car feel quicker when the tank is empty?
Here are some other thoughts. Unless you intend to re-install the system when you sell your car, it will destroy the resale value. Also, the A/C system is used in all seasons and not just summer. It works in conjunction with the defroster to un-fog windows.
From a financial perspective, for the money you’d spend to remove and reinstall the system, you could easily go Stage 1 and attain a meaningful boost in performance.