The candy house is where the witch lives. She's a nuisances and you can get rid of her.
First, go ahead and taste the house. Don't bother with the gingerbread children (who were likely once children) or the candy canes—eat the roof of house itself. You get points for this and it is a good way to tell if the witch is home or not. If you here a remark from someone in the house (the classic "nibble, nibble, little mouse"), you know she is home. If nothing is said, the house is empty and safe to enter. If you do here the witch, don't go inside unless you feel like becoming one of the gingerbread children yourself.
When the house is empty, you can walk in. There are two items of interest in the house. First, inside of the cupboard is cheese. You can use the cheese near the end of the game when you encounter the rat in the Leprechaun cave. The other item of interest is on the table the Witch's bed. The note says to "think backward". It is referring to the name of the Gnome on the Gnome island.
If you hang out in the Witch's house long enough, she will come home. If/when you hear her, go into the bedroom and wait there—you can rid yourself of the Witch. Once she shows up, she'll start talking about "who" she's going to eat for dinner and begin preparing her cauldron. When she is standing on the step in front of the cauldron., walk out of the bedroom and get behind her. Then, use the action hand to push her in to the cauldron. After that, you don't have to worry about her again. You also get points for this.