Generally this should be divided into two parts. First part is grammatical errors and typos, things that are just objectively wrong - there should be as little of those as possible. There are cases though when a purposeful alteration can achieve some specific, desired effect. This is especially relevant to dialogue, where you can purposefully insert grammatical mistakes to represent someone's specific quirks.
Apart from that, you can sometimes skirt that line between correct and purely wrong, especially if you're a narrator with personality, to get some specific effect such as tension (the easiest example I can think of is repeatedly using 'and' to list something out, which normally should be used once, maybe twice in a particularly long sentence). This is particularly useful in supernatural stories, but it's not limited to them.
With that said though, you should be careful with them, because unless you know very well what you are doing, it's very easy to make mistakes.
