Hello! After much trial and error of making my own 3D characters I decided to share my experience (since there is so little information online TT_TT)
This tutorial is meant for those who have a basic understanding of 3D modeling and rigging, Modeler functions. I use Blender and I will use it as an example (the basic principle is the same for other programs too). I won't teach how to sculpt a mesh or how to rig and weight paint. There are plenty of guides for it on Youtube. Instead I will show how to correctly set up the rig for export to Modeler so it is fully pose compatible and matches the original doll poses to (almost) a perfection. I will also include a small guide of final touches for materials and lightning.
P.S. the import format for Modeler should be FBX
There is a rigging guide in Japanese that talks in depth about bone structure. You can translate it in Google Translate or just flip through it to see the images (it also explains about separate head model, I will be using full body model):
The basic "perfect" posing
The thing about posing and 3D models in CSP is that all the official poses and the vast majority of asset poses are based on the original 3D doll shape (the first model above). All those poses will align perfectly in this case, but if you edit the body shape it naturally will get distorted - wider shoulders or hips, longer arms and legs and so on. For this reason the basic poses may disalign the position of wrists and ankles which is totally normal. Those of us who use edited shapes usually save new adjusted poses, that are perfect for the model, but not that perfect for the original shape.
It is important to understand what a perfect posing is in this case. What I refer to as "perfect" is the situation where if you made a 3D custom doll that matches the shape of the edited original 3D doll and apply the same pose to both, the angles would be pretty much identical (if you made a custom model with same shape as original model it would also be aligned the same on both):
So if you are making a completely new character, it may be beneficial to make it match the body shape of the CSP 3D doll you used until now in order to save the trouble of adjusting the old poses. If you never used any before and have no poses, good for you!! You can make whatever you want and create new poses for it. (People who use custom body shapes should understand this pain!!!!!)
The "human" rig of CSP
First rule: NAME YOUR BONES. There are gonna be many and you are bound to get lost. It is also useful to find the right bone in Modeler when adding accessories (you have a list to choose from and it is good to know what you need.
The root bone (first bone in the rig) is hips.
CSP uses a specific rig for its human dolls. It looks pretty much like this:
Notice how basic bones match in both dolls. If you want to create a HUMAN model that matches posing to CSP dolls (edited or not) it is important to keep the bone structure the same: Head, neck and 4 bones for spine, same amount of bones for hands and legs, toes included. The T pose of arms should be as straight as it can and so should be legs (unless you intend to have a basic standing shape distorted, like for animal creatures or demons with weird shapes). You can add as many additional bones as you want for other body parts like face, hair, clothes, additional limbs etc.. Important! Modeler shows "new" bones for disconnected bones by connecting them. Those are not bones!(the shoulder, hip and palm areas). Try to set your bones as closely in angle and proportion as you can. The one to one ratio for thigh and shin, upper arm and lower arm. You can make them distorted if that is your style, but keep in mind that it will also make poses even more distorted (like placing hand on knee may be incorrect once pose is applied). It is also important to keep axes in the same manner as the program will use them to decide rotation to some extent. This image lets to see the axes' orientation pretty well (most of them have no roll and use basic setup) and you can see more in detail bone by bone in the link at the beginning. The tricky part is spine and fingers.
(I added some additional bones for chest and face too)
The neck angle is pretty important. It is also important to get the shoulder angle close to the one in picture. I use reference images for getting the right angles. While they are important there is always a chance you will have to adjust them a bit after first test by a few angles depending on the model shape and distortion.
Some of references I have made for getting the right angles of different body parts:
Shoulder angle and the side view of hand. The length of a whole arm can be changed proportionally. the shoulder part length can differ, but the angle should remain about the same.
The spine side view starting from head down. Head +neck and 4 segments. You can see the length proportions of bones and neck angle. It is not important to get the back angle the same, but proportion in length matters for getting identical posing effect.
Hands (it's killing me!!!)
The trickiest part is hands! How to get those perfect hand poses finger tip to finger tip???? The secret is to keep true to original CSP proportions. you can change the width and length of entire hand, but not individual bones.
these are front and side views of bone positions in hands. Fingers should be aligned straight while keeping their respective bone to bone length proportion. In side view you can see the position of individual fingers. They pretty much are at same height except for pinky (P). The thumb requires a roll in axes that follows all segments. It is close to 45 degrees angle, but slightly rotated.
Thumb bone view from top and rotation (for left thumb)
I also found out that making the root of thumb bone closer to the wrist can help in nicer posing. So you can try to bring it closer to the wrist.
The final result is pretty much flawless!!
Adjustment and fixing errors
Modeler guide: https://www.clip-studio.com/site/gd_en/csm/userguide/csm_userguide/330_chara_plt/330_chara_plt_h3_0003.htm https://www.clip-studio.com/site/gd_en/csm/userguide/csm_userguide/330_chara_plt/330_chara_plt_h3_0003.htm
So now you have a "perfectly rigged" mesh(es), and imported it to modeler.
1. Height problems:
First you should set up height in Sketch tab. You can't change it after adding bones. You can add multiple bodies of same model with different clothes, shapes and bones. They all will follow the height adjusting proportion of first body. So you can have a human model and then you can have a slightly taller demonic animal leg version of it or maybe someone with high heels and the Modeler will do the math for you, but only if the first model bone mapping is done (keep in mind you cannot undo it). If you save it as a material and later reopen it in Modeler and decide to add another body, the proportion for new body will be lost. You need to redo everything from zero again to match all bodies.
2. The limbs are not aligned perfectly with used reference doll:
The arm thing can be easily fixed by slightly angling arms forward and the leg gap can be avoided by bringing feet closer.
3. Head problems:
So you have a complicated rig for face (more than just a head bone) and you see a mess of bones with no head bone (Modeler does that to non connected bones), but you try your luck and choose one of those head bones. Seems to work. All bones are detected properly. And then you go to SCP and find out that an eye or an ear can't be moved.
The fix for it is to choose the neck option for rigging instead of head (it will treat your character as multi part one, so don't use it as "a complete character".
4. Can't detect bones!!!!
So you have a similar case, you are using head option again, but instead of trying to find the missing head, you just clicked somewhere on the chest. It worked! Something lighted up, but... Now the next bones are just a mess.
The fix is the same as before. Choose neck option and redo the bone mapping.
5. Unmovable bones in CSP!
So you have a non standard model with some extra bones in limbs. You did the mapping and it did light up an entire limb like it should.
But when you loaded it in CSP there were parts that could not be selected or moved...
It is the same problem as a head one. There are extra bones and Modeler does not recognize the in between ones. In a leg with more than 4 bones, only four will be movable, including the last one (in this case a toe). Since there is no function like Neck one to teach Modeler where the basic leg bone chain should end you have to do a bit of workaround in original modeling program.
Add additional bone that plays as fake toes right after third bone down. Make sure to assign it a weight, since Modeler does not detect bones with no weights. Make the weight very small, so small that it basically does not deform anything (it's a fake anyway). It will be enough for Modeler to see it. In this case I added a 0.010 weights in Blender on feet.
Now when bone mapping in Modeler click on fake toes bone to teach it the correct mapping. In final result you will be able to move everything, but due to incorrect bones the basic poses wont affect anything below the fake toe. You can pose the feet (or any other custom part) manually after applying desired pose and then save that pose for future use. The saved poses will keep the custom part information and they also can be used on basic 3D dolls, it won't affect bones that are not there.
6. The weird angle after posing!?
You did a good job at setting everything up. You load the model and it looks damn good. Then you apply a pose and something goes wrong. Some body part completely changes its direction!
Fix: check your axes direction in original file. It may not go well with poses made using another custom model with different axes direction.
7. My model has no textures.
Make sure to export texture when you are exporting FBX file.
Setting up textures
Once you import model to Modeler you may notice that some of textures are not as you intended.
And if you load model to CSP it is too dark with black shadows (you need to enable light source in 3D settings first, as custom characters do not have a light on default):
So there is one last step for setting up characters and it is setting up materials.
First enable light in view settings (so you see shadows, but they won't be on default in CSP) and then head to materials folder.
In materials you can choose your materials one by one. It is a good idea to have them named beforehand. Each mesh is going to have it's separate material, so if you have many meshes it is possible you will have many materials. Click on the material and go to character information palette. You can set up the main color (if it's not an image texture), the specular light (the shiny part) and the diffuse part (the shadow part). 0 means black and 1 means full color. R G B stand for red, blue and green. By combining them together you can create dark, light and saturated or dim colors of your choice. In my case I want a pure white, which is all colors at 1. click on numbers and change them accordingly.
It is worth to note, that lightning in Modeler and CSP are different. Modeler has it lighter, while CSP has it darker. You may have to readjust it after first test.
For those who want to add some accessories (non movable objects on body) it is a must to have a rig with named bones, otherwise you gonna suffer a guess which bone is which quiz.
The basic setup for accessories is to set them at the center of both origin point and word center before exporting for Modeler. This way you will have easier time time adjusting their position and angle.
First go to accessories tab and load your accessory (I'll do the earrings). You can set up their materials the same way as it was done for body. Then choose the bone from list to which you want the object attached. Since I want my earrings to move with the head I choose the head bone.
At first try you may loose your object as it will randomly change location and angle. It inherits the rotation and position of bone root, completely messing up original setup. In orientation settings reset everything to zero to fix rotation. Then use move object tool to move them to desired position and further adjust the angle. Once done snip a photo to finish the setup. The hidden accessories won't be shown on load, but they can be enabled form 3D accessories tab.
And that's it for this guide! I hope it helped you to make your own well posed custom characters <3