homemade chilli oil

Yes, you should be concerned. Botulism is obviously very rare, and most people never get exposed to dangerous amounts of it even if they follow unsafe practices. Nevertheless, simply heating/boiling a mason jar does not sterilize it, it only pasteurizes it, so this is an unsafe practice, especially when you add the element of room-temperature storage.

Low-acid food needs to be pressure-canned. Period. It's been discussed on this site and it's covered by the FDA. Here's a relevant snippet:

What is the Best Way to Prevent Botulism?

• The control of foodborne botulism is based almost entirely on thermal destruction (heating) of the spores or inhibiting spore germination into bacteria and allowing cells to grow and produce toxins in foods. To prevent foodborne botulism:
• Use approved heat processes for commercially and home-canned foods (i.e., pressure-can low-acid foods such as corn or green beans, meat, or poultry).
• Discard all swollen, gassy, or spoiled canned foods. Double bag the cans or jars with plastic bags that are tightly closed. Then place the bags in a trash receptacle for non-recyclable trash outside the home. Keep it out of the reach of humans and pets.
• Do not taste or eat foods from containers that are leaking, have bulges or are swollen, look damaged or cracked, or seem abnormal in appearance. Do not use products that spurt liquid or foam when the container is opened.
• Boil home-processed, low-acid canned foods for 10 minutes prior to serving. For higher altitudes, add 1 minute for each 1,000 feet of elevation.
• Refrigerate all leftovers and cooked foods within 2 hours after cooking (1 hour if the temperature is above 90 °F).
• One of the most common causes of foodborne botulism is improperly home-canned food, especially low-acid foods such as vegetables and meats. Only a pressure cooker/canner allows water to reach 240 to 250 °F, a temperature that can kill the spores.

To summarize, in order to do this safely you need to do any or all of the following things (preferably all. in case you accidentally don't do one of the others correctly):

1. Acidify or pressure-can the low-acid ingredients.
2. Refrigerate the jars.
3. Boil it again before consuming. (You don't need pressure-canning temperatures at this stage).

(Note that I am assuming "crushed chilli" is coming from fresh chillies and not dried ones. If you are using 100% dry ingredients then it is probably safe although not 100% risk-free - see the answer to dry garlic in oil --> botulism risk?)

P.S. Regarding expiration, the oil is either safe or it isn't. It's safe if it's been properly pressure canned, otherwise you should assume it isn't safe and boil it as per #3 above. If it's been properly pressure-canned, then it should be good until/unless the oil goes rancid, which is more of a function of storage conditions (temperature, light) than time, and which you'll definitely be able to taste and probably be able to smell.

I have seen chili oil go rancid with and without heated oil. Neither way protects against the botulism toxin. Please note that this answer has been completely changed with now correct information (thanks to Aaronut and SAJ14SAJ):

updated

Home canning methods don't kill the botulism spores (thanks Aaronut for pointing that out). You'd need to use pressure canning techniques (temperature of 121C for at least 3 minutes) to be be as safe as possible. Otherwise it's recommended that Oils infused with fresh garlic or herbs should be acidified and refrigerated.

The FDA's Bad Bug Book provides details as well as the following recommendation:

• Because the botulism toxin is destroyed by high temperatures, home-canned foods are best boiled for 10 minutes before eating.

Finally the FDA has this fact sheet specifically covering information on how to deal with this bug.

Cooking oils boil at temperatures hundreds of degrees higher than needed to kill botulism. Heat the oil and seasonings to the required temperature in the oven or on stove then make sure the container you will store the oil in is sterilized. After you pour the oil into the containers it is not different from any commercial oil you would buy that has been opened . As for expiration date, you'd have to research the specific oil you have and the expiration date it had when you bought it.