It might be better than not receiving mail — but not by much: Outlook Express keeps collecting emails and refuses to delete any. The best you can do is move mail you do not want to folders you never inspect.
Chances are, Outlook Express is not mad at you or trying to preserve any single email you get. Most probably, the Deleted Items folder has been damaged — and it can be repaired.

Fix Outlook Express When Deleting Messages Does Not Work

To make Outlook Express delete messages again:

Go to the Deleted Items folder in Outlook Express.
Move all messages you may want to keep to your Inbox or another folder.
Shut down Outlook Express.
Open your Outlook Express store folder.
Delete the "Deleted Items.dbx" file.
Restart Outlook Express.
Make sure you empty the Deleted Items folder frequently (say, once a week).