Yes, it is perfectly fine to mix your fabrics. I would use batting. Enjoy.

I also like to use corduroy in a rag quilt, love the change in texture it provides. I don't use batting, just 3 layers of flannel. That way I cut all 3 layers the same size, don't have to do any quilting X's, and have more layers to rag.

I'm making my first rag quilt and am using 3 layers of flannel instead of using batting. Is there any limit on block size to not have to sew the Xs through the blocks? I'm doing a 4 patch block with 4" finished squares alternated with a solid 8" finished block. My thought is that it will be fine, but wanted to confirm that.

I have yet to make a rag quilt but want to someday so I enjoyed reading all the comments and I love your quilt that is shown. I like all the bright colors you have put into it.
Thanks for asking this question and thanks to everyone else who commented.
Pat