Oracle Corner

Before the cost optimizer, the optimizer used the heuristic rules to rewrite the query. Rule optimizer used to perform simple transformation: like indistinct merging of the sub-queries into the outer query, removing the redundant groups from views if influenced by the outer predicate and similar.

Cost optimizer compares the costs of different transformations and opts for the lowest one.
It makes more complex transformation. This is the list of the cost transformations introduced in 10g+ databases:

Sub-query unnesting

• Transformed as inline views
  

Join and predicate transformation

• Join predicate can be pushed into the view
• Groups can be pulled out of sub-query
• Group can be pushed into the sub-query
  
• Expensive predicates are pulled up
  
• (11g+) Early evaluation of the group by operators
  
• (11g+) Pull out common joins in similar UNION ALL

For more details, please read "Cost-Based Query Transformation in Oracle" by Rafi Ahmed, Allison Lee, Andrew Witkowski, Sinesh Das, Hong Su, Mohamed Zait and Thierry Cruanes